Design an Online Judge like LeetCode
Problem
Design a platform where users browse coding problems, write a solution in one of about 15 languages, and submit it to be compiled and run against hidden test cases. Each submission gets a verdict (accepted, wrong answer, time limit exceeded, runtime error, compile error) along with runtime and memory usage. The platform also hosts timed contests with a live leaderboard. It has 5 million monthly active users, sees about 1 million submissions per day, and during a weekly contest 50,000 participants submit in bursts that peak near 500 submissions per second. User code is untrusted and must never escape its sandbox.
- How many languages must be supported, and do we need per-language time limit multipliers?
- What is the maximum time and memory limit per test case, and how many test cases does a problem have?
- How quickly should users see a verdict: a few seconds normally, and what is acceptable during contests?
- Should the "Run" button (custom input, not graded) share capacity with real submissions?
- How is the contest leaderboard scored: number solved, then penalty time?
- Do we need plagiarism detection between contest submissions?
Items in bold are existing requirements. The rest are suggestions. Confirm with the interviewer whether they are relevant first.
- Browse problems with statements, examples, and constraints.
- Submit code in a chosen language and receive a verdict with runtime and memory.
- Run untrusted code in an isolated sandbox with enforced time, memory, and syscall limits.
- Host timed contests with a live leaderboard.
- Run code against custom input without grading it.
- Keep each user's submission history and show acceptance rates per problem.
Items in bold are existing requirements. The rest are suggestions. Confirm with the interviewer whether they are relevant first.
- 1 million submissions per day is about 12 per second on average; contests peak at 500 per second.
- Strict isolation: no network access, no access to other submissions or test data beyond stdin, bounded CPU, memory, processes, and disk.
- Typical verdict latency under 5 seconds; under 30 seconds at contest peak.
- A submission runs about 50 test cases for a total of roughly 2 CPU seconds, so contest peak needs about 1,000 busy CPU cores, plus headroom.
- Fair and reproducible timing: measured runtime should vary by less than 10% between runs of the same code.
- No submission may be lost, even if a judge worker crashes mid-run.
A web tier serves problems and accepts submissions, storing the code and enqueueing a judge job on a durable queue partitioned by priority. Stateless judge workers pull jobs, compile and run the code inside a locked-down sandbox against test cases cached locally from object storage, and write the verdict back. Verdicts are pushed to the browser over a WebSocket, and contest results update a leaderboard held in a sorted set.
Loading diagram...
Submit. The user clicks Submit. The submission API checks rate limits (for example, one submission per 5 seconds per user), stores the source code and metadata with status queued, and publishes a job { submissionId, problemId, language } to the judge queue. It returns the submission ID immediately.
Judge. A judge worker leases the job, fetches the source, and ensures it has the problem's test cases cached locally (keyed by problem version). It compiles the code in a sandbox, then runs each test case in a fresh sandboxed process with the problem's time and memory limits, comparing output with the expected output or a custom checker. It stops at the first failing test, records the verdict, peak memory, and max runtime, and acknowledges the job.
Deliver the verdict. The worker publishes a verdict event. The notification gateway, which holds the user's WebSocket, pushes the result to the browser. If the socket is closed, the browser falls back to polling GET /submissions/:id.
Update the leaderboard. For contest submissions, the leaderboard service consumes verdict events. On the first accepted verdict for a problem, it computes the user's new score and penalty and updates the sorted set.
Accepts submissions, enforces limits, and serves submission history.
- Stateless service in Go or Kotlin behind a load balancer; PostgreSQL for submissions; S3 for large source files.
API
POST /submissionswith{ problemId, language, code, contestId? }returns{ submissionId }.GET /submissions/:idreturns{ status, verdict, runtimeMs, memoryKb, failedTest? }.POST /runwith{ problemId, language, code, input }returns output without grading.
Data Model
submissions(id PK, user_id, problem_id, contest_id, language, code_ref, status, verdict, runtime_ms, memory_kb, created_at)with an index on(user_id, problem_id, created_at).
Tradeoffs
Judging asynchronously behind a queue, rather than holding the HTTP request open, keeps the API responsive during contest spikes and lets judge capacity lag demand briefly without failing requests.
Durable queue that buffers submissions and isolates contest traffic from everyday traffic.
- RabbitMQ or SQS with separate queues: contest, regular, and run.
Scalability
During a contest, contest submissions get dedicated workers, and the "Run" queue is throttled first when capacity is short. Queue depth is the autoscaling signal for judge workers.
Tradeoffs
Jobs are acknowledged only after the verdict is written, so a crashed worker's job is redelivered. Judging is deterministic and verdict writes are idempotent by submission ID, so at-least-once delivery is safe.
Compile and run untrusted code in isolation and decide the verdict.
- Workers on dedicated VMs; each run inside gVisor or Firecracker microVMs, or a minimal container with seccomp, cgroups, and namespaces (isolate or nsjail).
Diagram
Loading diagram...
Scalability
Workers pin each sandbox to a dedicated CPU core and run one test at a time per core, which keeps timing stable. At contest peak, about 1,000 cores are busy, so the pool pre-scales to about 1,500 cores 30 minutes before a scheduled contest instead of relying on reactive autoscaling. Test data is cached on local SSD so a 50-test run does not repeatedly download files.
Tradeoffs
A plain Docker container shares the host kernel, so one kernel exploit could escape it. Layered defenses (no network, read-only root filesystem, seccomp syscall filter, cgroup limits on memory and process count, and a non-root user) inside a microVM or gVisor reduce that risk at a startup cost of tens of milliseconds, which is small next to compile time. CPU time is measured rather than wall time, so a noisy neighbor does not cause false time limit verdicts; wall time is still capped at a multiple of the limit to catch sleep abuse.
Serves problem statements and stores the hidden test data.
- PostgreSQL for problem metadata, a CDN for statements, S3 for versioned test archives.
Data Model
problems(id, slug, title, statement_ref, difficulty, time_limit_ms, memory_limit_mb, checker_ref, test_version)- Test archive at
tests/{problemId}/{version}.tar.zst, readable only by judge workers.
Tradeoffs
Versioning test archives means a fix to a test case does not change verdicts for submissions already in flight, and rejudging can target a specific version. Test data is never served to browsers, so hidden cases stay hidden.
Maintains contest rankings in real time.
- Redis sorted sets per contest, persisted to PostgreSQL at contest end.
Data Model
- Sorted set
contest:{id}with memberuser_idand a composite scoresolved * 10^9 - penalty_seconds, so a higher score ranks higher. - Hash
contest:{id}:user:{uid}storing per-problem attempts and accepted time.
Scalability
50,000 participants in a sorted set use only a few megabytes, and ZREVRANGE for a page of 50 rows takes microseconds. The rendered top-100 page is cached for 5 seconds, so thousands of viewers refreshing does not touch Redis.
Tradeoffs
Encoding solved count and penalty into one numeric score allows a single sorted set instead of sorting in application code. The leaderboard is eventually consistent with verdicts by a second or two, which contests accept; a freeze in the last hour is a product rule implemented by hiding updates, not by stopping them.