Design a Feature Flag Service
Problem
Design a service that lets engineers turn features on and off without deploying code. A flag can be on or off globally, targeted at specific users or segments (such as country or plan), or rolled out to a percentage of users. Changes made in a dashboard must reach every running application within a few seconds, and every change must be recorded in an audit log. The service supports 1,000 customer organizations, 50,000 flags in total, and 500,000 application instances that evaluate flags on every request, about 5 million evaluations per second across the fleet.
- Do applications evaluate flags locally, or call a remote API on each evaluation?
- Are clients server-side services, browsers, mobile apps, or all three?
- What must happen when the flag service is unreachable: last known values, or hard-coded defaults?
- Must a percentage rollout be sticky (the same user always gets the same answer)?
- Do we need flag values beyond booleans, such as strings or JSON configuration?
- Do we need analytics on how often each variation is served, for experiments?
Items in bold are existing requirements. The rest are suggestions. Confirm with the interviewer whether they are relevant first.
- Create flags with variations and targeting rules (user lists, attribute rules, percentage rollouts).
- Evaluate a flag for a given user context in application code through an SDK.
- Propagate flag changes to all running applications within seconds.
- Keep an audit log of every flag change with who changed it and when.
- Support environments (development, staging, production) with separate flag states.
- Report evaluation counts per flag and variation, and flag flags that have not been evaluated in 30 days.
Items in bold are existing requirements. The rest are suggestions. Confirm with the interviewer whether they are relevant first.
- 5 million evaluations per second with evaluation latency under 1 ms, which rules out a network call per evaluation.
- Changes reach 99% of connected instances within 5 seconds.
- 500,000 instances hold open streaming connections.
- Flag configuration per environment is small: 50 flags at about 2 KB each, roughly 100 KB per environment.
- Applications must keep working with last known values if the flag service is completely down.
- Percentage rollouts are deterministic and sticky per user, with no stored per-user assignment.
Flag definitions are stored in a relational database and edited through a management API that writes an audit entry and bumps a version for the environment. SDKs download the full ruleset for their environment at startup, evaluate every flag locally in memory, and hold a streaming connection to a fan-out tier that pushes changes as they happen. A CDN-backed snapshot endpoint serves the initial download and acts as a fallback, while SDKs batch evaluation counts and send them asynchronously for analytics.
Loading diagram...
Change a flag. An engineer sets the new-checkout flag to a 10% rollout in production. The management API validates the rule, writes the new flag version and an audit entry in one transaction, increments the environment version, and publishes { env, flagKey, version } to the change bus.
Propagate. The snapshot builder regenerates the environment's full JSON ruleset and uploads it to the CDN under a versioned key. The streaming fan-out tier pushes a patch event with the new flag definition to every SDK connected for that environment. Each SDK applies the patch only if its version is newer than the one it holds.
Evaluate. In the request path, application code calls client.variation("new-checkout", user, false). The SDK evaluates rules in memory: individual targets first, then attribute rules in order, then the percentage rollout, which hashes flagKey + salt + userKey into a bucket from 0 to 99999. No network call is made.
Start up and recover. On startup, the SDK fetches the latest snapshot from the CDN, then opens its stream. If the stream disconnects, the SDK keeps serving the last ruleset, reconnects with jittered backoff, and polls the CDN snapshot every 30 seconds until the stream is back.
Source of truth for flags, rules, environments, and the audit trail.
- Stateless REST service backed by PostgreSQL.
API
POST /projects/:p/flagswith{ key, variations, defaultVariation }creates a flag.PATCH /projects/:p/envs/:e/flags/:keywith a list of semantic patch instructions (add rule, set rollout, turn off) and an optional comment.GET /projects/:p/envs/:e/flags/:key/historyreturns the audit trail.
Data Model
flags(project_id, key, variations JSON, created_by, archived)flag_env_state(project_id, env_id, key, on, targets JSON, rules JSON, fallthrough JSON, version)audit_log(id, project_id, env_id, key, actor, before JSON, after JSON, comment, created_at)
Tradeoffs
Writes are rare (a few per second across all customers), so a single relational database with strong consistency and transactions is simpler and safer than a distributed store. Updates use optimistic concurrency on version so two engineers editing the same flag cannot silently overwrite each other.
Library embedded in customer applications that holds the ruleset in memory and evaluates flags locally.
- Native SDKs per language (Java, Go, Node, Python) sharing a common evaluation spec and test suite.
Diagram
Loading diagram...
Scalability
Local evaluation costs microseconds and scales with the customer's own fleet, so 5 million evaluations per second never touch our servers. The SDK only holds rules for its own environment, typically under 1 MB.
Tradeoffs
Remote evaluation would keep rules private and simplify SDKs, but it adds a network round trip and a hard dependency to every request. Local evaluation trades that for the need to keep many SDK implementations behaving identically, which is enforced with a shared conformance test suite. Client-side SDKs in browsers are the exception: they receive only evaluated results for the current user, so targeting rules and other users' IDs are never exposed.
Holds long-lived connections to every SDK and pushes changes in real time.
- Server-Sent Events over HTTP/2, served by an event-driven Go or Rust service.
Scalability
A tuned node holds about 50,000 idle connections, so 500,000 instances need around 10 to 20 nodes plus headroom. Each node subscribes to the change bus for all environments and keeps an in-memory map of environment to connections. A single change touches only the connections of one environment.
Tradeoffs
Server-Sent Events are chosen over WebSockets because traffic is one-way, they work through proxies as plain HTTP, and reconnect with Last-Event-ID is built in. Pure polling is simpler but would need a 5-second interval from 500,000 instances (100,000 requests per second) to meet the propagation target.
Builds a full, versioned ruleset per environment and serves it from the edge.
- Bus consumer that writes JSON snapshots to object storage, fronted by a CDN with short TTLs.
Scalability
A deploy that restarts 50,000 instances at once is served entirely by the CDN, so a startup storm never reaches the flag store.
Tradeoffs
Shipping the whole environment rather than per-flag lookups keeps SDK startup to one request and makes the snapshot self-consistent, so a rule that references a segment always ships with that segment.
Receives batched evaluation counts from SDKs for usage analytics and experiments.
- Ingestion endpoint writing to Kafka, aggregated into ClickHouse.
Data Model
flag_evaluations(env_id, flag_key, variation, minute, count)
Tradeoffs
SDKs summarize counts in memory and flush every 30 seconds, dropping data if the collector is down, because analytics loss is acceptable but blocking evaluations is not. Sending one event per evaluation would mean 5 million events per second for little added value.