Design Twitter
Problem
Design a microblogging service where users post short tweets (up to 280 characters, optionally with media), follow other users, and read a home timeline of tweets from accounts they follow. Users can like and retweet, search all tweets by keyword within seconds of posting, and see trending topics for their region. The service has 250 million daily active users who post about 500 million tweets per day. Some accounts have over 100 million followers, and a single news event can drive tweet rates 5 to 10 times above normal.
- Is the home timeline reverse chronological, ranked, or both as separate tabs?
- How quickly must a new tweet appear in followers' timelines and in search results?
- How many accounts does a typical user follow, and how many accounts have more than 1 million followers?
- Should search cover all tweets ever posted, or recent tweets first with older tweets available more slowly?
- Are trends global, per country, or per city, and how often should they update?
- Do we need to handle replies and threads in this design, or only top-level tweets?
Items in bold are existing requirements. The rest are suggestions. Confirm with the interviewer whether they are relevant first.
- Post a tweet with text and optional media.
- Follow and unfollow users.
- View a home timeline of tweets from followed accounts.
- Like and retweet tweets.
- Search tweets by keyword with new tweets searchable within seconds.
- Show trending topics per region.
- View any user's profile timeline.
Items in bold are existing requirements. The rest are suggestions. Confirm with the interviewer whether they are relevant first.
- 500 million tweets per day is about 6,000 per second on average and 30,000 or more during spikes.
- Timeline reads dominate: about 250 million DAU x 20 loads = 5 billion per day, near 60,000 per second average and 200,000 at peak.
- Home timeline loads in under 200 ms at p99; tweets reach followers' timelines within 5 seconds.
- Tweets searchable within 10 seconds of posting.
- Tweet storage: 500 million x about 300 bytes = 150 GB per day of text and metadata, about 55 TB per year before replication; media is stored separately.
- High availability for reads; eventual consistency for counts and timelines is acceptable.
Tweets get time-ordered Snowflake IDs and are stored in a sharded tweet store, then published to an event stream that drives three independent consumers: timeline fan-out, a real-time search indexer, and a trend detector. Home timelines are precomputed lists of tweet IDs in an in-memory cache for ordinary accounts, while tweets from accounts with very large followings are merged in at read time. Search uses an in-memory inverted index for recent tweets backed by a larger on-disk index for the archive, and trends come from a streaming job that compares current term frequency to a baseline.
Loading diagram...
Post a tweet. The client sends POST /tweets. The tweet service gets a 64-bit Snowflake ID (timestamp, worker ID, sequence), writes the tweet to the tweet store sharded by tweet ID, adds it to the author's profile timeline, and publishes a TweetCreated event. It returns once the write is durable.
Fan out. The fan-out service reads the author's followers from the social graph. For an author below 1 million followers it inserts the tweet ID into each active follower's home timeline in the timeline cache. For an author above that threshold it does nothing more: the tweet is already in the author's profile timeline.
Read the home timeline. The timeline service reads the user's cached timeline (up to 800 tweet IDs), then fetches the recent tweet IDs of the high-follower accounts the user follows (typically a handful) from their profile timelines, and merges the lists by ID, which sorts by time. It hydrates tweets from the tweet cache, filters out deleted tweets and muted accounts, and returns a page of 20 with a cursor equal to the last tweet ID.
Search. The search indexer tokenizes each tweet and appends it to the realtime index within seconds. A query fans out to all realtime index partitions for recent results and to the archive index for older ones, then merges by a relevance score that weights recency and engagement.
Trends. The trend detector extracts hashtags and noun phrases from each tweet, counts them in sliding 5-minute windows per region, and compares each count with the term's baseline. Terms with the largest acceleration become trends.
Creates tweets and stores them durably, along with likes and retweets.
- Stateless service; sharded MySQL or Manhattan-style key-value store for tweets; Redis for counters.
API
POST /tweetswith{ text, mediaIds?, replyTo? }returns{ tweetId }.POST /tweets/:id/likeandPOST /tweets/:id/retweet.GET /users/:id/tweets?maxId=returns a profile timeline.
Data Model
tweets(tweet_id PK, author_id, text, media_ids, reply_to, created_at, deleted)sharded bytweet_id.user_tweets(author_id, tweet_id DESC)for profile timelines.likes(tweet_id, user_id)with counts aggregated asynchronously intotweet_counts(tweet_id, likes, retweets).
Tradeoffs
Snowflake IDs are generated locally without coordination, are roughly time-ordered, and fit in 64 bits, so timelines can sort and paginate by ID alone. A database auto-increment would be a single bottleneck, and UUIDs are not sortable. Like counts are approximate and updated asynchronously so a viral tweet does not hot-spot a single row.
Pushes tweet IDs to followers' home timelines and stores those timelines in memory.
- Kafka consumers; Redis Cluster storing a capped list per user, replicated three ways.
Diagram
Loading diagram...
Scalability
At 6,000 tweets per second and an average of a few hundred followers, fan-out produces roughly 1 to 2 million cache inserts per second, spread across many shards. Only users active in the last month get cached timelines, which keeps memory to a few hundred gigabytes of IDs.
Tradeoffs
Without the threshold, one tweet from a 100 million follower account would take minutes to fan out and delay every other tweet in the queue. Pulling those few accounts at read time adds a small, bounded merge to each read. The threshold is tuned by measuring fan-out lag against read cost.
Maintain an inverted index of tweets and answer keyword queries.
- Earlybird-style in-memory inverted index (or Lucene/Elasticsearch) partitioned by tweet ID; separate archive clusters on SSD.
Data Model
- Posting lists
term -> [tweetId...]appended in ID order, so the newest tweets are at the end and can be scanned first.
Scalability
The realtime index holds roughly the last 7 days and is partitioned by tweet ID ranges so every partition indexes an even share of the write stream; queries fan out to all partitions and merge. Each partition is replicated for read throughput. The archive index is built in batch from the tweet store and queried only when realtime results are insufficient.
Tradeoffs
Partitioning by term would let a query hit one shard, but popular terms during a news spike would overload one partition. Partitioning by document spreads writes evenly at the cost of scatter-gather on every query, which is acceptable because each partition answers in a few milliseconds.
Finds terms whose usage is accelerating unusually fast in a region.
- Flink or Spark Structured Streaming over the tweet stream; Count-Min Sketch for approximate term counts.
Data Model
trends(region, computed_at, rank, term, tweet_volume)refreshed every minute.
Scalability
The stream is keyed by (region, term) so counts are aggregated in parallel. A Count-Min Sketch bounds memory per window when the number of distinct terms explodes, and a heavy-hitters structure keeps the top candidates per region.
Tradeoffs
Ranking by raw count would always surface the same perennial terms, so the score compares current rate with a trailing baseline (for example, the same window over the last week). The detector also drops terms dominated by few accounts or newly created accounts, which blocks most coordinated manipulation.
Stores follow relationships in both directions.
- Sharded MySQL or a graph-optimized key-value store, with both directions materialized.
Data Model
following(user_id, followee_id, created_at)sharded byuser_id.followers(user_id, follower_id, created_at)sharded byuser_id.
Tradeoffs
Storing both directions doubles writes but makes "who do I follow" and "who follows me" single-shard reads, which fan-out and timeline merges need constantly. Follower lists of the largest accounts are paged in batches so fan-out never loads 100 million rows at once.