CDN & Caching

Concept#

The fastest way to serve a request is to not do the work at all, to already have the answer sitting somewhere close to the requester. That's the entire premise of caching: trade a little staleness risk for a large reduction in latency and, just as importantly, in load on the systems that would otherwise have to do the real work every single time.

The cache hierarchy: layers between a user and your origin server#

A request for a resource typically has several opportunities to be served from a cache before it ever reaches the server that computed the original answer:

Browser cache     → the user's own device already has this exact
                     response cached from a previous visit (governed
                     by Cache-Control headers).
        ↓ (miss)
CDN / edge cache   → a Content Delivery Network node, physically
                     close to the user, has a cached copy from
                     serving a DIFFERENT user's earlier request.
        ↓ (miss)
Application cache  → an in-memory or Redis-style cache sitting in
                     front of your application logic / database.
        ↓ (miss)
Database           → the actual origin of truth. Every layer above
                     exists specifically to avoid reaching this.

Each layer that can serve a hit saves real, cumulative cost: latency (no round trip to a far-away origin), origin load (a busy database never even sees the request), and bandwidth. A well-designed system aims for the overwhelming majority of requests to be satisfied by the outer layers, with the database only handling requests nothing else could answer.

CDNs: caching that's also geography#

A Content Delivery Network is a network of servers ("edge nodes" or "points of presence") distributed globally, positioned physically close to end users. When a user in Tokyo requests a static asset your origin server hosts in Virginia, a CDN edge node in or near Tokyo can serve a cached copy directly, cutting a transoceanic round trip down to a local one. This is why CDNs are the default first move for static assets (images, JS/CSS bundles, videos): the content rarely changes, and the latency win from geographic proximity is enormous and essentially free once set up.

Modern CDNs go further than static files, many can cache full HTML pages (for content that's the same for every visitor) and even run compute at the edge, but the foundational use case remains: put a copy of content physically near the person requesting it.

Cache-Control: the actual mechanism, not just a suggestion#

Caching behavior in browsers and CDNs is governed by the Cache-Control HTTP header, which is a real, precise instruction, not a hint:

Cache-Control: public, max-age=31536000, immutable
  public      → cacheable by shared caches (CDNs), not just the browser
  max-age     → how many seconds this response is considered FRESH
  immutable   → tells the browser not to even revalidate on refresh, this exact URL will NEVER change its content
 
Cache-Control: no-cache
  → MUST revalidate with the origin before using a cached copy
    (confusingly, this does NOT mean "don't cache", it means
    "cache, but check first")
 
Cache-Control: no-store
  → the actual "don't cache this at all" directive
    (used for sensitive, per-user, or highly dynamic responses)

A widely-used pattern combines a long max-age with cache-busting: name static assets with a content hash in the filename (app.a1b2c3.js instead of app.js), set immutable, max-age=31536000 (effectively "cache forever"), and when the content changes, ship a new filename with a new hash. The old cached file is now simply irrelevant, nobody requests it anymore, and the new filename is a guaranteed cache miss everywhere, so there's no invalidation to coordinate at all. This sidesteps cache invalidation rather than solving it, which is exactly why it's so effective.

Cache invalidation: "there are only two hard things in computer science"#

The famous line, cache invalidation, naming things, and off-by-one errors, earns its place because invalidation genuinely has no universally correct answer; it's always a tradeoff between staleness and cost:

TTL (time-to-live):  cache entry expires automatically after N seconds.
                      Simple, no coordination needed. Tradeoff: data can
                      be stale for up to N seconds after a real change.
 
Explicit purge:       actively tell the cache "this specific entry is
                      now invalid" the moment the underlying data
                      changes. Fresher, but requires the write path to
                      know about and successfully reach every cache
                      layer that might be holding a stale copy.
 
Cache-busting:        (see above) sidesteps invalidation for STATIC,
                      content-addressed assets by changing the URL
                      itself whenever content changes.
 
Write-through:        write to the cache AND the origin at the same
                      time, keeping them in sync as writes happen, trades write latency for read freshness.

The hard part in real systems isn't picking one of these, it's that different data has genuinely different staleness tolerance (a product price can tolerate a few seconds of staleness; a bank balance mid-transfer cannot), and a system usually needs several of these strategies simultaneously, applied deliberately per type of data rather than as one blanket policy.

Distributed Cache Sharding & Consistent Hashing#

When scaling a cache layer horizontally across a cluster of nodes (e.g. Memcached, Redis Cluster, CDN Points of Presence), simple modulo hashing (hash(key) % N) causes a catastrophic cache stampede when any node joins or leaves the pool (remapping $(N-1)/N$ of all cached keys to wrong servers).

Consistent Hashing maps both cache servers and data keys onto a $360^\circ$ circular hash ring. When a server node joins or crashes, only $K/N$ keys migrate to the clockwise successor node, preserving $1 - 1/N$ of the entire cluster cache hit ratio. Virtual Nodes (V-Nodes) interleave tokens across the ring to smooth standard deviation and eliminate hot spot imbalance.

Try It#

A news site caches article pages at the CDN with Cache-Control: public, max-age=300 (5 minutes). An editor fixes a factual error in a published article. What will readers see, and for how long, under this configuration, and what would you change if the correction needs to go out immediately?

Implement It Yourself#

A minimal in-memory LRU (Least Recently Used) cache, the actual eviction mechanism behind most application-tier caches, including how libraries like Redis implement maxmemory-policy allkeys-lru:

class LRUCache {
  constructor(capacity) {
    this.capacity = capacity;
    this.cache = new Map(); // Map preserves INSERTION order in JS
  }
 
  get(key) {
    if (!this.cache.has(key)) return undefined;
    const value = this.cache.get(key);
    // Re-insert to mark as MOST recently used (moves it to the end)
    this.cache.delete(key);
    this.cache.set(key, value);
    return value;
  }
 
  set(key

This is the real mechanism, not a simplification of it, the "move to the end on access" trick using a JS Map's insertion-order guarantee is exactly how you'd implement O(1) LRU behavior without a separate doubly-linked-list structure, which is what most textbook LRU implementations reach for in languages without an ordered map primitive.

Under the Hood#

CDN and browser caching both key off the same Cache-Control mechanism covered in depth, with Next.js-specific caching layers stacked on top of it, in Next.js Caching & Revalidation, that topic covers the framework-level caching (data cache, full route cache) that sits between your application code and the raw HTTP caching semantics described here.

Common Mistakes#

1. Using no-cache when you meant no-store#

Cache-Control: no-cache // ❌ this DOES still cache, it just always revalidates first

no-cache is one of the most commonly misread HTTP headers, despite the name, it permits caching; it just requires revalidation with the origin before serving the cached copy. For genuinely sensitive data that must never be cached at all (auth tokens, personal financial data), the correct directive is no-store.

2. Setting a long max-age on content that changes unpredictably#

Cache-Control: public, max-age=86400 // ❌ 24-hour cache on a live sports score endpoint

A long max-age on frequently-changing data means users can be served hours-stale data with no mechanism to correct it short of an explicit purge. Match max-age to the data's actual staleness tolerance, a live score needs a max-age of seconds, not a day, or should bypass caching (or use a much shorter TTL plus client-side polling) entirely.

3. Relying on cache invalidation as the primary strategy for static assets#

// ❌ Deploying app.js repeatedly to the SAME filename and trying to purge every CDN edge node on every deploy
Cache-Control: public, max-age=31536000
// filename never changes → stale JS served to users until every edge purges successfully

For genuinely static, content-addressed assets, cache-busting (a content hash in the filename) sidesteps invalidation entirely and is far more reliable than trying to actively purge every CDN edge node on every deploy, a purge can fail silently on some subset of nodes, while a new filename is a guaranteed miss everywhere with zero coordination required.

Best Practices#

• Use content-hashed filenames plus long max-age, immutable for static assets, this sidesteps invalidation entirely rather than trying to solve it.
• Match cache duration to actual staleness tolerance per data type, not a single blanket policy across the whole site, a product description and a live inventory count have very different tolerances.
• Reserve no-store for genuinely sensitive or per-user data, and use no-cache (revalidate-before-use) rather than a long max-age for data that changes unpredictably but does benefit from conditional caching.
• Build an explicit purge path into publishing workflows for content that occasionally needs immediate propagation (corrections, urgent updates), rather than relying solely on max-age expiry.
• Cache at every layer that makes sense, not just one, browser, CDN, and application-tier caching solve different parts of the latency/origin-load problem and typically all pay for themselves independently.

Performance Tips#

• The latency win from a CDN is largely a geography win, the physical distance between a user and the server serving their request often dominates over raw server processing time for anything CDN-cacheable.
• An LRU (or similar) eviction policy on an application cache matters more as cache size shrinks relative to the working set, with generous memory, eviction policy rarely matters; under real memory pressure, LRU's "recently used data is likely to be used again" assumption is what keeps hit rates high.
• Cache hit rate, not raw cache size, is usually the metric to optimize, a cache that's technically large but has a low hit rate (because of poor key design or overly short TTLs) delivers little real benefit.