Design a Content Delivery Network
Problem
Design a content delivery network that sits in front of customer websites and serves their static and cacheable content (images, scripts, video segments, API responses) from servers close to end users. The network has about 200 points of presence (PoPs) around the world, and must route each user to a nearby, healthy PoP. Customers configure caching rules per domain and must be able to purge content globally within a few seconds. The network serves 50 million requests per second and 100 Tbps of peak egress, and must keep serving when a PoP or a customer origin fails.
- What mix of content do we serve: small web assets, large video files, or dynamic API responses?
- Do customers bring their own TLS certificates, or do we issue them?
- How fast must a purge take effect, and do customers purge by URL, by prefix, or by tag?
- Should stale content be served when the origin is down?
- Do we need edge compute, such as running customer code at the PoP?
- Is the customer origin a server we can reach directly, or object storage?
Items in bold are existing requirements. The rest are suggestions. Confirm with the interviewer whether they are relevant first.
- Serve cached content from the PoP nearest to the user.
- Fetch from the customer origin on a cache miss and cache according to rules and HTTP headers.
- Let customers configure per-domain caching rules.
- Purge content by URL, prefix, or tag globally within seconds.
- Terminate TLS at the edge with customer certificates.
- Serve stale content when the origin is unreachable, if the customer allows it.
- Provide per-customer traffic and cache hit metrics.
Items in bold are existing requirements. The rest are suggestions. Confirm with the interviewer whether they are relevant first.
- Handle 50 million requests per second and 100 Tbps of peak egress across about 200 PoPs, or roughly 250,000 requests per second and 500 Gbps per PoP on average.
- Route each user to a nearby healthy PoP, keeping median round trip time under 30 ms for most users.
- Global purge completes within 5 seconds at p99.
- Cache hit ratio above 95% for typical static content, to protect origins.
- Survive the loss of a PoP or an origin without user-visible errors where cached content exists.
- Config changes reach every PoP within a minute.
Users reach the network through anycast IP addresses, so BGP routes each connection to a nearby PoP, and the network withdraws a PoP's routes when it is unhealthy. Inside a PoP, load balancers spread requests over cache servers by consistent hashing on the URL, and misses go first to a regional shield tier before reaching the customer origin. A global control plane distributes configuration, certificates, and purge commands to every PoP, while logs flow back for metrics and billing.
Loading diagram...
Serve a request. The user resolves cdn.example.com, which points (by CNAME) at one of the network's anycast addresses. Their TCP or QUIC connection lands at the nearest PoP as decided by BGP. The PoP load balancer hashes the request URL to pick a cache server, which terminates TLS with the customer's certificate, applies the domain's rules, and builds a cache key (host, path, selected query parameters and headers). On a hit with a fresh object, it responds immediately.
Cache miss. The edge server sends the request to the regional shield PoP responsible for that origin. If the shield also misses, it fetches from the origin, collapsing concurrent requests for the same object into one origin fetch. The response flows back, and each tier stores it according to Cache-Control headers or the customer's override rules.
Purge. A customer calls the purge API. The control plane records the purge, assigns it a sequence number, and fans it out over a persistent push channel to every PoP. Each cache server marks matching objects invalid (by URL, prefix, or tag) and acknowledges. The API reports completion when all PoPs acknowledge, typically in 1 to 3 seconds.
Steers each user to a nearby PoP that has capacity and is healthy.
- Anycast: the same IP prefixes announced via BGP from every PoP.
- Authoritative DNS with latency-based answers as a secondary mechanism for large file and video hostnames.
Scalability
Anycast spreads load automatically, since each user's network picks the shortest path. When a PoP is overloaded or failing, it withdraws or deprioritizes its announcements and traffic shifts to neighbors within seconds, without waiting for DNS TTLs to expire.
Tradeoffs
DNS-based routing gives precise control over which PoP each resolver uses, but it routes by the resolver's location rather than the user's and is slow to react because of cached DNS answers. Anycast reacts fast and needs no client cooperation, but a routing change can move a long-lived connection mid-stream. Using anycast for most traffic and DNS steering for long video sessions combines both strengths.
The server in each PoP that terminates connections and serves cached content.
- A high-performance proxy such as NGINX, Varnish, or a custom Rust server.
- Memory for hot small objects, NVMe SSDs for the larger working set.
Diagram
Loading diagram...
Data Model
- Cache index entry:
cache_key_hash,disk_location,size,expires_at,tags[],purge_sequence.
Scalability
Consistent hashing of URLs across servers in a PoP means each object is stored on one server, so the PoP's total cache capacity is the sum of its servers rather than one server's worth. Very popular objects are additionally replicated to several servers so a single machine is not overwhelmed.
Tradeoffs
Hashing within the PoP increases hit ratio but adds an internal hop when the load balancer cannot hash directly. Caching each object on every server avoids that hop but wastes most of the storage on duplicates. The hashing approach wins because hit ratio drives both origin load and latency.
A middle cache layer between edge PoPs and customer origins.
- Selected large PoPs act as shields, each responsible for a set of origins.
Scalability
Without a shield, a miss at each of 200 PoPs would hit the origin separately. Funneling misses through one shield per region reduces origin requests to a handful per object and raises the effective hit ratio for long-tail content.
Tradeoffs
The shield adds latency to cache misses and is another layer to operate. For customers with powerful origins and little long-tail content, it can be disabled. Serving stale content while revalidating, and when the origin returns errors, keeps sites up through origin outages.
Holds customer configuration and pushes configuration, certificates, and purges to the edge.
- A replicated config database (PostgreSQL) as the source of truth.
- A fan-out tree of persistent connections (or a global pub/sub system) to every PoP.
API
PUT /zones/:zone/rulesupdates caching rules.POST /zones/:zone/purgewith{ urls?, prefixes?, tags? }returns{ purgeId }.GET /purges/:purgeIdreturns completion status.PUT /zones/:zone/certificatesuploads a certificate.
Data Model
zones(zone_id, customer_id, hostnames[], origin, rules_version)purges(purge_id, zone_id, selector, sequence, created_at, acked_pops)
Scalability
Configurations are versioned snapshots that PoPs cache locally, so the edge keeps serving with its last known config if the control plane is unreachable. Purges are small messages, and the fan-out tree delivers them to 200 PoPs in one or two hops.
Tradeoffs
Purging by marking entries invalid with a sequence number is far faster than deleting files from disk, and the space is reclaimed later by normal eviction. Tag-based purge needs an index from tag to objects on each server, which costs memory, but it lets customers invalidate all pages that show a product with one call.
Collects request logs for customer metrics, billing, and operations.
- Local log agents at each PoP that batch and compress logs, streaming to Kafka in central regions.
- A columnar store such as ClickHouse for dashboards and billing aggregates.
Scalability
At 50 million requests per second, raw logs are huge, so PoPs pre-aggregate common metrics (requests, bytes, hit ratio per customer per minute) and ship full logs only for customers who pay for them, sampled for everyone else.
Tradeoffs
Pre-aggregation at the edge makes dashboards cheap and timely but limits ad hoc queries. Shipping every log line centrally would allow any query but cost a large share of egress bandwidth, so full logs are an opt-in feature.