System design
Classic design prompts with compact model answers, plus the building blocks interviewers probe: caching, CAP and PACELC, load balancing, queues, consistent hashing, retries, and SLOs.
Answer out loud or on paper before revealing. If your answer misses a key point, revisit the linked guide or explainer.
Design a URL shortener like bit.ly.
Reveal model answer
Requirements: create a short code for a long URL, redirect fast, optional custom aliases and expiry, basic click analytics; reads dominate writes. Estimates: say 100M new links a month is about 40 writes/s, and at 100:1 reads about 4k redirects/s average with peaks several times higher; 7 base62 characters give about 3.5 trillion codes, and 6 billion links over five years at roughly 500 bytes each is about 3 TB. Components: a stateless API behind a load balancer, a key-value or relational store mapping code to URL, a cache (and CDN) in front of reads, and an async pipeline for click events. Code generation: base62-encode a unique ID from a sequence or ID service, or hand out pre-allocated ranges to each app server; random codes need a collision check. Data model: code (primary key), long_url, owner_id, created_at, expires_at. Redirect with 302 or 307 if you want every click to reach you for analytics, 301 or 308 if you prefer browser caching. Bottlenecks and trade-offs: hot links (cache), analytics writes (queue and batch, never on the redirect path), abuse and malware links (rate limits, scanning), and enumerable sequential codes (shuffle or add randomness).
WHAT A STRONG ANSWER COVERS
- Clarifies requirements and the read-heavy ratio before drawing boxes.
- Back-of-envelope estimates for QPS, key space, and storage.
- A concrete code-generation scheme and how collisions or coordination are handled.
- Cache-heavy read path; analytics off the critical path.
- 301 versus 302 and its effect on analytics.
- Abuse, expiry, and predictable-code concerns.
RED FLAGS
- Hashing the URL with MD5 and truncating without handling collisions.
- Writing a click row synchronously on every redirect.
- Jumping to sharding before estimating that a single store might suffice.
LIKELY FOLLOW-UPS
- How would you support custom aliases without races?
- How would you serve redirects with low latency worldwide?
SOURCES
Design a distributed rate limiter for an API gateway.
Reveal model answer
Requirements: limit requests per key (API key, user, IP) per time window, configurable per tier and endpoint, add only a millisecond or two of latency, stay accurate across many gateway instances, and fail in a defined way. Estimates: at 100k requests/s each request does one counter operation, so the store must handle 100k+ atomic ops/s with sub-millisecond latency, which points at an in-memory store such as a Redis cluster sharded by key. Algorithm: a token bucket (capacity = burst, refill rate = sustained limit) stored as tokens plus last-refill timestamp and updated atomically in a Lua script, or a sliding-window counter that weights the previous window's count, which Cloudflare found accurate enough at a tiny memory cost. Data model: key = limiter:{tenant}:{route}, value = counters, TTL slightly longer than the window. Response: 429 with Retry-After and remaining-quota headers. Bottlenecks and trade-offs: a round trip to Redis on every request (mitigate with local pre-aggregation that syncs periodically, trading some accuracy), hot keys for huge tenants, multi-region (per-region limits or async sync rather than a global lock), and fail-open versus fail-closed when the store is down. Pair it with concurrency limits and load shedding, which protect the service rather than enforce fairness.
WHAT A STRONG ANSWER COVERS
- Clear requirements: keying, limits, latency budget, failure mode.
- Algorithm choice with trade-offs (token bucket, fixed window, sliding window log or counter).
- Atomic updates in a shared store; no read-then-write races.
- Where it runs: gateway, sidecar, or library, and the extra hop it costs.
- Fail-open versus fail-closed and multi-region accuracy trade-offs.
RED FLAGS
- Per-instance counters behind a round-robin load balancer.
- A sliding-window log for high-volume keys without considering memory.
- No answer for what happens when Redis is unavailable.
LIKELY FOLLOW-UPS
- How would you give enterprise customers burst capacity?
- How would you rate limit globally across three regions?
SOURCES
Design a chat system with one-to-one and group messages.
Reveal model answer
Requirements: send and receive messages in near real time, history, delivery and read receipts, presence, offline delivery via push, multiple devices per user; ordering must be correct within a conversation. Estimates: 50M daily users sending 40 messages a day is 2B messages/day, about 23k writes/s average and several times that at peak, with tens of millions of concurrent connections. Components: clients hold a WebSocket to a fleet of stateful gateway servers; a session registry maps user to gateway; a message service validates, assigns a per-conversation sequence number, persists, then fans out through a pub/sub layer to the gateways holding recipients' connections, falling back to push notifications for offline devices. Data model: messages partitioned by conversation_id (bucketed by time for very active channels) and clustered by a time-sortable message ID, which is how Discord stores trillions of messages; plus per-user per-conversation read cursors. Bottlenecks and trade-offs: reconnect storms after a gateway deploy (jittered reconnects, resume from last sequence), huge groups (fan-out on read or dedicated handling), hot partitions for busy channels, at-least-once delivery with client-side de-duplication by message ID, and end-to-end encryption limiting server-side features like search.
WHAT A STRONG ANSWER COVERS
- Persistent connections (WebSocket) and a way to route to the right gateway.
- Persist first, then fan out; client message IDs for idempotent sends.
- Per-conversation ordering via sequence numbers, not wall clocks.
- Partition storage by conversation; plan for hot channels.
- Offline delivery, multi-device sync, and resume after reconnect.
RED FLAGS
- Polling the database every second from every client.
- Global ordering across all conversations.
- Ignoring what happens when a gateway node restarts with 500k open sockets.
LIKELY FOLLOW-UPS
- How do you deliver messages to a 100k-member channel?
- How does a client sync after being offline for a week?
SOURCES
Design a news feed (home timeline) for a social network.
Reveal model answer
Requirements: post content, follow users, see a reverse-chronological or ranked feed of followed accounts with low latency; feed reads vastly outnumber posts, and a few seconds of delay is acceptable. Estimates: 200M daily users opening the feed 10 times a day is about 23k feed reads/s average, while posts are far fewer, but each post may need to reach hundreds or millions of followers. Components: a post service and store, a social graph service, a fan-out service, a timeline cache, and a feed service that hydrates post IDs into full posts. Core choice: fan-out on write pushes each new post ID into every follower's precomputed timeline (fast reads, expensive for accounts with millions of followers), while fan-out on read merges followed accounts' recent posts at request time (cheap writes, slow reads). Twitter's well-known answer was a hybrid: fan out on write into in-memory timelines for most accounts, and merge posts from very high-follower accounts at read time. Data model: posts(id, author_id, created_at, body), follows(follower_id, followee_id), timeline:{user_id} as a capped list of post IDs. Bottlenecks and trade-offs: celebrity fan-out, inactive users (skip precomputing), deletes and privacy changes (filter at hydration time), ranking cost, and cursor pagination over a moving feed.
WHAT A STRONG ANSWER COVERS
- Read-heavy, eventually consistent; latency matters more than freshness.
- Fan-out on write versus fan-out on read, and the hybrid for celebrities.
- Store IDs in timelines; hydrate from a post cache.
- Capped timelines and skipping inactive users to bound cost.
- Cursor-based pagination and handling deletes.
RED FLAGS
- Only fan-out on write, without addressing accounts with millions of followers.
- A giant SQL join across follows and posts on every feed load at scale.
- Copying full post bodies into every follower's timeline.
LIKELY FOLLOW-UPS
- How would you add ML ranking without blowing the latency budget?
- How do you remove a deleted post from millions of timelines?
SOURCES
Compare caching strategies: cache-aside, read-through, write-through, write-behind. Where do you put caches?
Reveal model answer
Cache-aside (lazy loading) is the most common: the app reads the cache, on a miss loads from the database and populates the cache, and on writes updates the database and deletes the key. Read-through and write-through move that logic into the cache layer; write-through writes to cache and database synchronously, keeping the cache warm at the cost of write latency, while write-behind acknowledges after the cache write and persists asynchronously, which is fast but can lose data. Caches live at several layers: browser and CDN via HTTP headers, an in-process cache (fast but each instance has its own copy and starts cold), and a shared cache such as Redis or Memcached. A cache becomes a dependency: if the database can only survive with a high hit rate, a cold or failed cache causes an outage, so test with the cache disabled and use soft TTLs and request coalescing.
WHAT A STRONG ANSWER COVERS
- Cache-aside flow and why writes usually delete rather than update the key.
- Write-through versus write-behind durability trade-off.
- Local versus shared caches: coherence and cold starts.
- TTLs with jitter, negative caching, request coalescing.
- The cache-as-crutch risk when the backend cannot handle a cold cache.
RED FLAGS
- "Just add Redis" without an invalidation story.
- Write-behind for money or other data you cannot lose.
- No plan for a cold cache after a deploy or failover.
LIKELY FOLLOW-UPS
- What happens when a hot key expires under heavy load?
- How do you size a cache and pick an eviction policy?
SOURCES
How do you keep a cache consistent with the database after writes?
Reveal model answer
The usual approach is to write to the database, then delete the cache key so the next read reloads it; deleting is safer than setting because two concurrent writers updating the cache can leave the older value last. There is still a race: a reader can load the old value, a writer commits and deletes, and the reader then writes the stale value back, so keep TTLs as a safety net and consider versioned values or a short delayed second delete. For multi-service setups, drive invalidation from the database change stream (CDC or an outbox) so a failed delete call cannot leave the cache stale indefinitely. Decide per data type how much staleness is acceptable: a product description can be minutes old, an account balance should not be cached that way at all.
WHAT A STRONG ANSWER COVERS
- Write database then delete key; prefer delete over update.
- The read-populate race and TTL as a backstop.
- Versioning or compare-and-set to avoid writing stale values.
- CDC or outbox-driven invalidation for reliability.
- Staleness budget per data type.
RED FLAGS
- Updating the cache before the database transaction commits.
- No TTL at all, so one missed invalidation lasts forever.
- Assuming invalidation is solved because each write path deletes the key.
LIKELY FOLLOW-UPS
- How do you invalidate CDN-cached pages after a CMS update?
- How would you invalidate a list cache when one item changes?
SOURCES
Explain the CAP theorem and PACELC. How do they affect real design decisions?
Reveal model answer
CAP says that during a network partition a distributed system must choose between consistency (every read sees the latest write, linearizability) and availability (every request to a non-failed node gets a response); partition tolerance is not optional, so "pick two of three" is misleading. Brewer himself noted that partitions are rare and the choice can be made per operation. PACELC, from Daniel Abadi, adds the everyday case: if there is a Partition, trade Availability against Consistency; Else, trade Latency against Consistency, because synchronous replication costs latency even with no failures. In practice you choose per feature: account balances and unique usernames want consistency, while likes counts and feeds tolerate staleness for lower latency and higher availability.
WHAT A STRONG ANSWER COVERS
- C in CAP means linearizability, not the C in ACID.
- P is a fact of networks; the real choice is C versus A during a partition.
- PACELC adds the latency versus consistency trade-off in normal operation.
- Choices are per operation or per data type, not per system.
RED FLAGS
- "We chose CA."
- Labeling a database "CP" or "AP" without discussing its configuration.
- Confusing CAP consistency with ACID consistency.
LIKELY FOLLOW-UPS
- Where does a single-primary PostgreSQL with async replicas sit in PACELC?
- What does quorum (R + W > N) buy you?
SOURCES
How does load balancing work, and which algorithm would you choose?
Reveal model answer
A load balancer spreads requests across healthy backends, either at layer 4 (TCP connections, cheap and protocol-agnostic) or layer 7 (HTTP-aware, so it can route by path or header, terminate TLS, and retry). Round robin is simple but ignores that requests differ in cost, so backends drift apart; least connections or least outstanding requests adapts to load, and weighted or utilization-aware policies did best in Google's experience. Health checks remove failed backends, and connection draining or a lame-duck state lets instances finish in-flight work during deploys. Long-lived connections such as WebSockets or gRPC streams stick to one backend, so balancing them requires periodic reconnection or client-side balancing, and session affinity should be avoided unless state truly lives on the instance.
WHAT A STRONG ANSWER COVERS
- L4 versus L7 trade-offs.
- Round robin versus least-outstanding versus weighted by utilization.
- Active and passive health checks; draining on deploy.
- Long-lived connections and sticky sessions skew load.
- Client-side balancing and subsetting at large scale.
RED FLAGS
- Round robin as the answer for every workload.
- Health checks that only test that the process is up, not its dependencies or readiness.
- Making the load balancer a single point of failure.
LIKELY FOLLOW-UPS
- How would you balance WebSocket connections after scaling out?
- How do you do a zero-downtime deploy behind a load balancer?
SOURCES
When do you put a queue between services, and how do you handle back-pressure?
Reveal model answer
A queue decouples producers from consumers, absorbs bursts, and lets slow or failing work retry without blocking the user request, at the cost of eventual consistency, at-least-once delivery that requires idempotent consumers, and harder debugging. A queue does not add capacity: if producers outpace consumers for long enough, the backlog grows until messages are too old to matter and recovery takes hours. Handle that with back-pressure (bounded queues, rejecting or slowing producers as depth grows), autoscaling consumers on queue age rather than depth, per-tenant isolation or shuffle sharding so one noisy customer cannot fill the queue, TTLs that drop stale work, and dead-letter queues for poison messages. Monitor the age of the oldest message as the primary SLI.
WHAT A STRONG ANSWER COVERS
- Benefits: decoupling, buffering, retries; costs: async semantics and duplicates.
- Queues hide overload; they do not remove it.
- Back-pressure through bounded queues, throttling, and load shedding.
- Isolation per tenant or workload; DLQs for poison messages.
- Alert on oldest-message age, not just depth.
RED FLAGS
- Unbounded in-memory queues.
- Retrying a poison message forever and blocking the partition.
- Assuming exactly-once delivery.
LIKELY FOLLOW-UPS
- How do you preserve per-customer ordering while scaling consumers?
- How would you drain a 10-hour backlog after an outage?
SOURCES
What is consistent hashing, and why is it used?
Reveal model answer
With naive hash(key) mod N, adding or removing one node remaps almost every key, which empties caches and forces mass data movement. Consistent hashing places nodes and keys on a hash ring, and each key belongs to the next node clockwise, so a membership change only moves the keys between the affected node and its neighbor, roughly 1/N of them. Plain rings distribute load unevenly, so systems give each physical node many virtual nodes (tokens), as Amazon's Dynamo did, which smooths the distribution and lets bigger machines take more tokens. Replication is usually done by storing a key on the next few distinct nodes on the ring. Alternatives include rendezvous hashing and fixed hash slots, as in Redis Cluster's 16384 slots.
WHAT A STRONG ANSWER COVERS
- mod N remaps nearly everything on resize.
- Ring placement: only about 1/N keys move per change.
- Virtual nodes for balance and heterogeneous capacity.
- Replicas on the next distinct nodes clockwise.
- Hot keys are not solved by consistent hashing alone.
RED FLAGS
- Claiming consistent hashing guarantees even load without virtual nodes.
- Confusing it with consistency models.
LIKELY FOLLOW-UPS
- How would you handle a single extremely hot key?
- How do fixed hash slots compare with a ring?
SOURCES
How do you make retries safe across a chain of services?
Reveal model answer
Retries are only safe if the operation is idempotent, so mutating calls carry a client-generated request token that each service records with the result and replays on duplicates, returning a semantically equivalent response. Retrying at every layer multiplies load: three attempts at each of five layers is up to 243 calls at the bottom, so retry at one layer (usually the edge or the caller closest to the failure), use capped exponential backoff with jitter, and cap retries with a token bucket or a retry budget, like Google's limit of retries to about 10 percent of requests. Set timeouts from downstream latency percentiles and propagate deadlines so callees stop working on requests the caller has abandoned. Never retry non-retryable errors such as validation failures.
WHAT A STRONG ANSWER COVERS
- Idempotency keys or request tokens with stored results.
- Retry amplification across layers; retry in one place.
- Exponential backoff with jitter and a retry budget.
- Timeouts and deadline propagation.
- Distinguish retryable (timeouts, 503, 429) from non-retryable errors.
RED FLAGS
- Fixed-interval retries with no jitter.
- Retrying POSTs without idempotency keys.
- Every client library and proxy configured to retry independently.
LIKELY FOLLOW-UPS
- When would a circuit breaker help, and what does it cost?
- How do you choose a timeout value for a downstream call?
SOURCES
How would you define SLOs and set up observability for a new service?
Reveal model answer
Start from what users experience: pick SLIs such as the fraction of requests that succeed and the fraction served faster than a latency threshold, measured as close to the user as practical, and set SLOs like 99.9 percent of requests successful over 28 days. The gap to 100 percent is the error budget, which tells you when to slow releases and focus on reliability; an SLA is the contractual promise with consequences and should be looser than the SLO. Instrument the four golden signals (latency, traffic, errors, saturation), use percentiles rather than averages, add distributed traces with propagated context (OpenTelemetry) and structured logs that share the trace ID. Page on symptoms that burn the error budget quickly, not on every cause like high CPU.
WHAT A STRONG ANSWER COVERS
- SLI versus SLO versus SLA.
- Error budgets drive release and reliability decisions.
- Golden signals; percentiles over averages.
- Metrics, traces, and logs correlated by trace ID.
- Symptom-based, actionable alerts (burn-rate alerting).
RED FLAGS
- "100 percent availability" as the target.
- Alerting on CPU above 80 percent as the primary page.
- Average latency as the only latency metric.
LIKELY FOLLOW-UPS
- How would you write a burn-rate alert for a 99.9 percent SLO?
- What is high-cardinality data, and why does it matter for metrics?
SOURCES