Design Pastebin

• POST /pastes with { content, visibility, expiresIn?, syntax? } returns { key, url }.
• GET /:key returns the rendered HTML page, or 404 if the paste is missing or expired.
• GET /raw/:key returns text/plain content.
• DELETE /pastes/:key deletes a paste (author only).
• GET /users/me/pastes?cursor= lists the author's pastes.

Peak traffic of a few thousand requests per second needs only a small fleet, and most reads never reach it because the CDN answers them. Bodies over 1 MB are uploaded with a presigned URL directly from the client to object storage, so large uploads do not tie up service instances.

Rendering syntax highlighting in the browser keeps the server fast and lets the stored body stay plain text. Server-side rendering would help search engine indexing of public pastes, but the added CPU per request is not worth it for a service whose pages are mostly unlisted.

• pastes: key (partition key), author_id, visibility, size_bytes, syntax, created_at, expires_at, object_key.
• pastes_by_author: author_id (partition key), created_at (sort key), key.
• expiry_index: expiry_bucket (partition key, one per minute), key, used by the sweeper.

At about 200 bytes per row, even a billion pastes is only around 200 GB of metadata. Random keys spread writes evenly across partitions. The expiry index is bucketed by minute so the sweeper reads one small partition at a time instead of scanning the table.

Keeping bodies out of the metadata store keeps rows small and cheap to replicate, at the cost of a second lookup per cold read. A relational database would also work at this size, but every query is a point lookup by key, and native TTL plus easy horizontal scaling make a key-value store the simpler long-term choice.

• Object key: pastes/<key>, with Content-Encoding: gzip and the paste key as metadata for orphan cleanup.

Object storage scales storage and request rate without operational work and replicates across zones for durability. Text compresses well, typically 3 to 5 times with gzip, which cuts the 18 TB per year estimate substantially. Lifecycle rules move pastes older than 90 days to an infrequent access tier.

Storing bodies in the database would remove a network hop, but 10 MB rows are a poor fit for most databases and inflate replication and backup costs. Object storage is far cheaper per gigabyte. Because the body is written before the metadata row, a crash between the two leaves an orphaned object rather than a paste that points at nothing, and a weekly job deletes objects with no metadata.

Pastes are immutable, so cached copies never go stale except through deletion or expiration. A viral paste is served almost entirely from CDN edges. The in-memory cache uses LRU eviction and only stores small bodies so a few large pastes cannot evict thousands of small ones.

The CDN cannot cache unlisted pastes any differently from public ones, which is acceptable because unlisted security relies on the unguessable key, not on hiding the response. Deleting a paste requires a CDN purge, which can take seconds to propagate. That short delay is accepted in exchange for offloading most reads.

Each run reads one expiry bucket, so its work is proportional to the number of pastes expiring that minute rather than to the total library. If the sweeper falls behind, nothing incorrect is served, because the read path checks expires_at itself.

Relying only on the database's native TTL would delete metadata but leave bodies in object storage and stale copies on the CDN. An explicit sweeper costs a small amount of code but cleans up all three in one place.