A rate limiter caps how many requests a client may make in a window. Put it at the API gateway, return 429 when the budget is exceeded, and pick from four classic algorithms that trade burst tolerance against memory and accuracy.
Place the limiter at the API gateway, the edge proxy every request already crosses, so rejected traffic never reaches expensive backends. In a distributed fleet, all gateway nodes must share counters, typically in Redis, or a client can dodge the limit by spreading requests across nodes.
Token bucket allows bursts up to bucket size and is the common default. Fixed-window counters are cheapest but let traffic spike to 2× the limit at window edges. Sliding-window logs are exact but store a timestamp per request in a Redis sorted set. Pick based on how much burst you can tolerate against how much memory you’ll spend.
Source: Alex Xu, System Design Interview Vol 1, Ch. 4
Answer to reveal the explanation. Nothing is scored.
1A client is being rejected by the limiter. What status code does it get?
429 tells the client it exceeded its budget and may retry later. A 503 means the server is overloaded, and a 403 means the client is never allowed.
2You must tolerate short bursts up to a fixed size but hold a steady average rate. Which algorithm?
Token bucket refills at rate r up to capacity b, so it permits bursts up to b while bounding the long-run rate.
3Why must gateway nodes in a fleet share limiter state?
With per-node counters a client hitting N nodes gets N× the budget. Shared state gives every node one view.