Design a News Feed
Problem
Design the home feed of a social network. Users follow other users and pages, and when they open the app they see a feed of recent posts from the accounts they follow, ranked by relevance rather than strictly by time. The feed must load in under 300 ms and support infinite scroll without showing duplicates or skipping posts. The network has 500 million daily active users, each following about 200 accounts on average; a small number of accounts have tens of millions of followers. Users create about 500 million posts per day and refresh their feed about 10 times per day.
- Is the feed purely from followed accounts, or does it also include recommended and sponsored content?
- How fresh must the feed be? Is it acceptable if a new post appears for followers within a few seconds?
- What signals can ranking use: engagement, affinity with the author, recency, content type?
- How far back should a feed go? Is the last few days enough for most users?
- Do we need to handle edits and deletes, and should a deleted post disappear from feeds immediately?
- Do inactive users need precomputed feeds, or can we build them when they return?
Items in bold are existing requirements. The rest are suggestions. Confirm with the interviewer whether they are relevant first.
- Show a feed of posts from followed accounts, ranked by relevance.
- Support paginated infinite scroll with no duplicates or gaps.
- New posts appear in followers' feeds shortly after publishing.
- Remove deleted posts and posts from unfollowed or blocked accounts.
- Mix in recommended or sponsored posts at controlled positions.
Items in bold are existing requirements. The rest are suggestions. Confirm with the interviewer whether they are relevant first.
- Feed load latency under 300 ms at p99.
- 500 million DAU x 10 refreshes = 5 billion feed reads per day, about 58,000 per second average and 150,000 at peak.
- 500 million posts per day is about 6,000 per second; with 200 followers each, naive fan-out is 1.2 million feed inserts per second.
- Post visibility in followers' feeds within 5 seconds for normal accounts.
- Feed storage: 500 post IDs of 16 bytes per active user is 8 KB, or about 4 TB for 500 million users, which fits in a cache cluster.
- Availability over consistency: a slightly stale feed is better than an error.
The system uses a hybrid fan-out. Posts from ordinary accounts are pushed at write time into each follower's precomputed feed list in a cache, while posts from high-follower accounts are pulled at read time and merged in. On read, the feed service gathers candidates from both sources, hydrates them with post and author data, scores them with a ranking model, and returns a page addressed by an opaque cursor tied to a snapshot of the ranked list.
Loading diagram...
Publish. A user creates a post. The post service writes it to the post store and emits a PostCreated event. If the author has fewer than 100,000 followers, fan-out workers read the follower list in pages and prepend (postId, authorId, createdAt) to each active follower's feed list, trimming each list to 500 entries. If the author is above the threshold, workers only append the post to the author's entry in the celebrity post index.
Read the feed. The client calls GET /feed. The feed service reads the user's precomputed list (about 500 candidate IDs), looks up which followed accounts are celebrities, and pulls their recent posts from the celebrity index. It removes posts from blocked or unfollowed authors, batch-fetches post content and counts from the post cache, and sends candidates with features to the ranking service.
Rank and paginate. The ranking service scores each candidate and returns an ordered list. The feed service stores this ranked list of IDs as a short-lived session snapshot (10 minutes), returns the first 20 hydrated posts, and returns a cursor { snapshotId, offset }. Scrolling reads the next 20 from the snapshot, so ranking changes cannot cause duplicates or gaps mid-session. Pull-to-refresh starts a new snapshot.
Consume post events and push post IDs into followers' feed lists.
- Kafka consumers in Java or Go, writing to a Redis Cluster feed cache.
Diagram
Loading diagram...
Scalability
Skipping inactive followers and routing high-follower accounts to the pull path cut the 1.2 million naive inserts per second substantially. Each feed write is a small pipelined list operation, and workers are partitioned by author so the stream scales horizontally.
Tradeoffs
Pure fan-out on write makes reads cheap but means one post from an account with 50 million followers triggers 50 million writes and delays everyone else's posts. Pure fan-out on read makes writes cheap but means every feed load queries 200 authors' timelines, which is too slow at 150,000 reads per second. The hybrid uses each where it is cheap.
Stores each active user's list of candidate post IDs.
- Redis Cluster, sharded by user ID, with replicas for read scaling.
Data Model
feed:{userId}list ofpostId:authorId:timestampentries, capped at 500.
Scalability
About 4 TB across the cluster for 500 million users. Users who have been inactive for 30 days have their lists evicted; when they return, the feed service rebuilds the list by pulling from followed authors' recent posts (a one-time cost).
Tradeoffs
Storing only IDs rather than full posts keeps the cache small and means edits, deletes, and like counts never need to be fanned out; post content is hydrated at read time from the post cache.
Assembles candidates, filters, hydrates, calls ranking, and paginates.
- Stateless service in Go or Java with aggressive parallel batch calls.
API
GET /feed?cursor=&limit=20returns{ posts: [...], nextCursor }.POST /feed/refreshinvalidates the session snapshot and returns a fresh first page.
Scalability
Every dependency call is batched (one multi-get for 500 posts, one for authors), and the service runs them in parallel, keeping the critical path to a few cache round trips plus ranking. If ranking times out, the service falls back to reverse chronological order rather than failing.
Tradeoffs
Offset pagination over a live ranked list shows duplicates as new posts arrive. Timestamp cursors do not work because the order is not chronological. A session snapshot of ranked IDs is cheap (500 IDs) and gives stable pagination.
Scores candidate posts for a given viewer.
- Two-stage ranking: a lightweight model trims 500 candidates to 100, then a heavier gradient-boosted tree or neural model scores the rest.
- Feature store (Redis or a dedicated store) holding viewer-author affinity, post engagement rates, and content type preferences.
Scalability
Features are precomputed offline and in streaming jobs, so scoring needs only lookups and model inference. Models run on CPU fleets sized for 150,000 requests per second times 100 candidates.
Tradeoffs
Ranking by predicted engagement increases time spent but can bury recent posts from close friends, so the final score blends model output with recency and diversity rules (no more than two consecutive posts from one author).
Durable storage of posts and a cache of hot post objects for hydration.
- Cassandra or a sharded MySQL for posts; Memcached for post objects and counters.
Data Model
posts(post_id PK, author_id, body, media_refs, created_at, visibility, deleted)posts_by_author(author_id, created_at DESC, post_id)backing the celebrity index and feed rebuilds.
Tradeoffs
Post IDs are time-sortable (Snowflake style), so recency comparisons need no extra lookup. Deletes set a tombstone; the feed service drops tombstoned posts during hydration rather than scrubbing every follower's feed list.