Design a Recommendation System
Problem
Design the personalized home feed of a video platform. When a user opens the app, show a ranked list of about 50 items chosen from a catalog of 10 million videos. The feed must load in under 200 ms, reflect what the user just watched within a few minutes, and work for brand new users with no history. The platform has 200 million daily active users who open the home feed several times a day.
- What is the business objective: clicks, watch time, satisfaction surveys, or retention?
- How often is new content added, and must new videos get exposure quickly?
- Are there hard constraints such as parental controls, regional licensing, or diversity rules?
- Can we use contextual signals such as device, time of day, and location?
- Do we need to explain recommendations (for example "because you watched X")?
- How do we evaluate a new model before full rollout?
Items in bold are existing requirements. The rest are suggestions. Confirm with the interviewer whether they are relevant first.
- Return a personalized ranked list of about 50 items for the home feed.
- Update recommendations based on actions within the last few minutes.
- Serve sensible recommendations to new users and surface new videos.
- Filter out content the user cannot watch (region, age rating, already finished).
- Keep the feed diverse rather than showing 50 similar items.
- Support A/B testing of models and ranking rules.
Items in bold are existing requirements. The rest are suggestions. Confirm with the interviewer whether they are relevant first.
- 200 million daily users opening the feed about 5 times each: 1 billion requests per day, about 12,000 per second, peaking near 40,000.
- Feed latency under 200 ms at p99, including all model inference.
- Interaction logging peaks around 1 million events per second (impressions, clicks, watch progress).
- Freshness: user features update within minutes; models retrain daily.
- Degrade gracefully: if ranking fails, serve a cached or popularity-based feed rather than an error.
Recommendation is split into two stages. Several cheap candidate generators (embedding nearest neighbors, co-watch graphs, trending, subscriptions) narrow 10 million items to about 1,000 candidates, and a heavier ranking model scores those candidates using features read from a low-latency feature store. Every impression and interaction is logged, joined with the features that were served, and used to retrain the models, closing the loop.
Loading diagram...
Serve a feed. The app calls GET /feed. The feed service loads the user's feature vector and recent history from the online feature store, then calls candidate generators in parallel: nearest neighbors to the user embedding, items co-watched with recent views, subscriptions, and regional trending. It unions about 1,000 candidates, removes ineligible ones, fetches item features, and sends the batch to the ranking model. Finally, it applies diversity rules, takes the top 50, and logs the impression with a request id.
Learn from interactions. The app streams impressions, clicks, and watch progress tagged with the request id. Streaming jobs update short-term features such as "last 10 videos watched" and "categories clicked this session" within seconds. All events land in the data lake.
Retrain. Daily, the training pipeline joins impressions with outcomes (did the user watch more than 50%?) and with the exact feature values logged at serving time, trains the ranking model and refreshed embeddings, validates them offline, and registers them. A new model receives a small slice of traffic in an A/B test before full rollout.
Orchestrates candidate generation, filtering, ranking, and fallbacks for each request.
- Java or Go service with parallel fan-out and strict per-call timeouts.
API
GET /feed?cursor=&context=returns{ items: [{ itemId, reason }], requestId, nextCursor }.POST /feedbackwith{ requestId, itemId, action }for explicit signals like "not interested".
Scalability
The service is stateless. Each dependency has a timeout (for example 30 ms for candidates, 60 ms for ranking); a generator that misses its deadline is skipped. Each user's last feed is cached so a ranking outage can fall back to it.
Tradeoffs
Computing the feed on request keeps it fresh and responsive to the last few minutes of activity. Precomputing every user's feed nightly would be cheaper per request but stale and wasteful, since many users never open the app on a given day.
Retrieves a few hundred plausible items from each of several sources.
- Two-tower model producing user and item embeddings, served from an approximate nearest neighbor index such as FAISS or ScaNN.
- Precomputed co-watch lists and trending lists in a key-value store.
Diagram
Loading diagram...
Scalability
The ANN index holds 10 million vectors of 128 dimensions (about 5 GB with float32, less with quantization), small enough to replicate fully on each node. Item embeddings are recomputed daily; new items get an embedding from their metadata immediately so they are retrievable on day one.
Tradeoffs
Multiple generators provide recall from different angles, and each is easy to add or remove. A single generator is simpler but fails silently for whole groups of users, such as new users with no embedding history, who rely on trending and onboarding choices.
Provides consistent features for both training and serving.
- Online store in Redis or a low-latency key-value store; offline store in the data lake as Parquet.
- Feast or a similar framework that defines each feature once for both paths.
Data Model
- User features:
user_id -> { embedding, top_categories, watch_count_7d, last_10_items, country, device }. - Item features:
item_id -> { embedding, age_hours, ctr_24h, avg_watch_fraction, language, rating }.
Scalability
Reads per request are one user row and about 1,000 item rows, fetched in batched multi-gets. Item features are hot and cached in the ranking servers' memory, so the store mostly serves user lookups.
Tradeoffs
Logging the served feature values and training on those logs avoids training and serving skew, at the cost of extra storage. Recomputing features for training from raw events is cheaper but often subtly differs from what was served, silently degrading the model.
Scores each candidate for the probability of meaningful engagement.
- A multi-task deep neural network predicting click, watch fraction, and like, served with GPU or optimized CPU inference (Triton or TorchServe).
Scalability
Candidates are scored in one batch per request. At 40,000 requests per second and 1,000 candidates each, that is 40 million item scores per second, spread across a GPU fleet with dynamic batching.
Tradeoffs
Optimizing a blend of watch time and satisfaction rather than raw clicks discourages clickbait. The weights of that blend are a product decision and are tuned through A/B tests on long-term retention, not offline metrics alone.
Builds labeled examples, trains models, and safely rolls them out.
- Spark for joining impressions with labels, a distributed training framework, a model registry, and an experimentation platform that assigns users to buckets.
Scalability
Training data is sampled: all positive examples and a fraction of negatives, reweighted to stay unbiased. Daily incremental training starts from yesterday's weights.
Tradeoffs
Daily retraining plus real-time features captures most freshness benefit at a fraction of the complexity of fully online learning. A small exploration share of traffic shows items the model is uncertain about so the system keeps learning instead of reinforcing its own past choices.