CDNs

The problem: an ocean between your users and your files

Picture a photo app whose server and photo files sit in California, with users all over the world. For a user in Sydney, every image crosses the Pacific: a round trip between continents takes about 150 ms.

That server has a second problem. Say the app shows about 10,000 photos a second, about 1 MB each: 10 GB a second leaving one place. One server's network sends about 1 GB a second, so that is ten servers' worth of network, mostly crossing oceans.

The obvious fixes fail: a faster server doesn't shorten the distance, and moving everything to Sydney makes everyone else slow.

How a CDN works

A content delivery network (CDN) is many edge servers in data centers around the world, each keeping copies of your files near the users. Your own servers become the origin, where the files really live. Each request goes to a nearby edge: the CDN's DNS answers with a nearby edge's address, or every location announces the same address (anycast) and internet routing delivers to the closest. Say a round trip to a nearby edge takes about 20 ms.

Most CDNs are pull CDNs. The edges start empty, and for each request:

1. If the edge has a fresh copy, that's a hit: it answers by itself, in about 20 ms.
2. If not, that's a miss: the edge fetches the file from the origin, keeps a copy and sends it on.
3. The origin's Cache-Control header sets how long the copy is kept, its TTL: max-age=86400 is one day, in seconds.

Six requests to edges in Sydney and London, both starting empty:

#  user in  asks for  that edge holds     result                     wait
1  Sydney   logo.png  nothing             miss: fetch, keep 1 day    20 + 150 = 170 ms
2  Sydney   logo.png  logo.png            hit                        20 ms
3  London   logo.png  nothing             miss: its own copy         20 + 150 = 170 ms
4  London   logo.png  logo.png            hit                        20 ms
5  Sydney   cat.jpg   logo.png            miss: fetch, keep 1 day    20 + 150 = 170 ms
6  Sydney   cat.jpg   logo.png cat.jpg    hit                        20 ms

Each edge fills on its own: Sydney's copy doesn't help London, so a file misses once per edge. A popular file costs the origin almost nothing: if 1,000 Sydney users open the logo in a day, that's 1 miss and 999 hits. A miss costs about the same as no CDN (170 ms against 150), so the hit rate is what matters. Many CDNs add a middle layer of caches (an origin shield), so a file missed at many edges is fetched from the origin about once.

The numbers: what the CDN saves the origin

Only misses reach the origin. At 10,000 photos a second of about 1 MB:

• 90% hits: 1,000 photos a second, 1 GB a second: one server's whole network.
• 95% hits: 500 a second, 0.5 GB a second: half of one. The edges send the other 9.5 GB.
• 99% hits: 100 a second, 0.1 GB a second.

At the busy hour, about 5× the average, that's 50 GB a second, and 95% hits leave 2.5 GB a second for the origin. The rate depends on the mix: the logo hits almost always, a photo few people see may expire before anyone asks again. State the rate you assume.

What to put on a CDN, and what not

An edge hands the same copy to everyone near it who asks for that address, so the rule is: the same bytes for many users, changing rarely.

• Through the CDN: images and icons, public product photos, video, scripts, stylesheets, fonts.
• From your servers each time: a bank balance, private messages, the checkout page with a cart, search results picked for one user. Each is made for one person or must be exact now: a shared copy could be stale or reach the wrong user.

Private files, such as a user's own photos, can still go through a CDN with signed URLs that expire (Files and Object Storage). Video streaming cuts videos into pieces a few seconds long, each a file the CDN caches (Design YouTube).

Changing a file that's cached everywhere

The edges don't know when the origin changes. Release a new app.js with a new page, and a user can get the new page but the old script from an edge: the page breaks. Three ways out:

• Wait for the TTL: simple, but with max-age=86400 some users get the old file for up to a day.
• Purge: tell the CDN to drop the copy at every edge. It takes a moment to reach them all, and it can't reach the copies in users' browsers, which keep the file by the same header.
• A new file name per version (the usual answer): put a short hash of the content in the name, like app.3f9a.js, and have the new page ask for it. No edge or browser has it yet, so it's fetched fresh, and the old app.2b71.js is never asked for again. A name's content never changes, so it can be kept for a year (max-age=31536000, 365 days); only the page itself gets a short TTL.

A TTL of zero is no fix: every request then goes on to the origin, and the CDN saves nothing.

Pull or push

A pull CDN fetches files on their first miss: no setup, and only what's asked for takes space. With a push CDN you upload files ahead of time and it spreads them to its edges.

Push fits a few big files wanted at a known time. Say a game's 2 GB update goes live at 9:00 and the CDN has 100 edges: with pull, every edge misses at once and 200 GB leave the origin together, over 3 minutes at 1 GB a second. Pushed the night before, it's waiting at every edge. Pull fits many files whose popularity you can't predict: pushing a photo app's new photos (say 1 TB a day) to 100 edges would copy 100 TB a day, most of it never viewed in that city.

Where CDNs show up in design interviews

Any design with images, video or users around the world gets a CDN. Say what goes through it (photos, video, scripts, styles) and what doesn't (the API: feeds, messages, anything per user); that it pulls, with TTLs from Cache-Control; the numbers ("at 95% hits the origin sends about 0.5 GB a second instead of 10"); and how versions change: a new file name per version.

Common interview questions on CDNs

What is a CDN, and when do you need one?

Edge servers around the world that keep copies of files near the users. You need one when users are far from your servers or file traffic outgrows their networks: a nearby hit takes about 20 ms instead of 150.

What's the difference between a push and a pull CDN?

A pull CDN fetches a file from the origin on an edge's first miss and keeps it for its TTL; a push CDN gets files uploaded ahead of time. Pull is the usual default; push suits big files wanted at a known moment.

How do you update a file that's cached on a CDN?

Give every version a new name, usually with a hash of its content, and point the page to it. Purging works too but takes time and misses browsers' caches; waiting means up to a full TTL of old copies.

Can a CDN serve private content?

Not as a shared copy: anything made for one user, like a balance or messages, comes from your servers. Private files that don't change, like a user's photos, can go through a CDN with signed URLs that expire.

Summary

• A CDN keeps copies of files on edge servers near the users; DNS or anycast picks a nearby edge.
• Pull: an edge fetches a file on its first miss and keeps it for the TTL. Each edge has its own copies.
• A hit takes about 20 ms, a miss about 170; at 95% hits the origin sends 0.5 GB a second instead of 10.
• Put there the same bytes for many users, changing rarely; give each version a new file name.