Swe Api Gateway Rate Limiting Patterns

Rate Limiting Is a Top-5 System Design Interview Topic in 2026

Rate limiting design questions (“design a rate limiter for an API gateway”) remain a fixture in mid-to-senior system design loops because the problem is small enough to whiteboard in 35-45 minutes but rich enough to expose gaps in distributed systems reasoning: clock skew, race conditions across multiple gateway replicas, and the tension between strict correctness and low latency. Unlike algorithm questions, there’s no single “right” data structure, so interviewers are grading your trade-off reasoning as much as your implementation.

The Four Core Algorithms

Token Bucket. A bucket holds up to N tokens, refilled at a fixed rate (e.g., 10 tokens/second). Each request consumes one token; if the bucket is empty, the request is rejected or queued. Token bucket allows short bursts up to the bucket capacity, which is usually the desired behavior for user-facing APIs. It requires only a counter and a last-refill timestamp per key, making it cheap to store in Redis with a Lua script for atomicity.

Leaky Bucket. Requests enter a fixed-size queue and are processed (leaked) at a constant rate. Unlike token bucket, leaky bucket smooths bursts into a strictly constant output rate, which is preferable when downstream systems (databases, third-party APIs) cannot tolerate bursty traffic at all, at the cost of added queuing latency.

Fixed Window Counter. Increment a counter per fixed time window (e.g., per calendar minute) and reject once the counter exceeds the limit. Simple and cheap but has a well-known boundary flaw: a client can send N requests at the end of one window and N more at the start of the next, achieving 2N requests in a much shorter interval than the window size, a bug interviewers explicitly probe for.

Sliding Window Log / Sliding Window Counter. Sliding window log keeps a timestamp per request and counts requests within the trailing window, which is accurate but memory-heavy at scale. Sliding window counter approximates this by weighting the previous window’s count proportionally to the overlap, which is what most production gateways (including Cloudflare and AWS API Gateway-style implementations) actually use because it balances accuracy against O(1) storage.

Distributed Correctness: Where Candidates Actually Fail

A rate limiter that works correctly on a single node commonly breaks once you have multiple gateway replicas behind a load balancer. The standard fix is centralizing counter state in Redis (or a similar low-latency store) and using atomic operations (INCR + EXPIRE, or a Lua script combining check-and-increment) to avoid race conditions where two replicas both read a stale counter and both allow a request that should have been the one that tips over the limit. Interviewers frequently ask “what happens if Redis is briefly unavailable” — a strong answer discusses fail-open vs fail-closed trade-offs explicitly rather than assuming Redis is always up.

Clock skew across gateway nodes is a second common trap: if window boundaries are computed from local system clocks rather than a synchronized source, different replicas can disagree on which window a request falls into, undermining fixed and sliding window approaches alike.

Where Rate Limiting Lives in the Request Path

Rate limiting at the API gateway layer (rather than inside each service) centralizes policy, avoids duplicating logic across services, and lets you reject abusive traffic before it consumes downstream compute. Most 2026 production stacks apply limits at multiple layers: coarse IP-based limits at the edge/CDN, authenticated per-user or per-API-key limits at the gateway, and sometimes fine-grained per-endpoint limits inside the service itself for expensive operations (e.g., search or export endpoints).

Comparison Table: Rate Limiting Algorithms

Interview Answer Structure

A strong answer sequences as: clarify requirements (per-user vs per-IP, burst tolerance needed), pick an algorithm with a stated trade-off reason, describe the storage layer (Redis with atomic Lua script), address the distributed race condition explicitly, and finally discuss fail-open/fail-closed behavior under partial outage. Candidates who jump straight to “I’ll use token bucket” without clarifying requirements first typically lose points even with correct implementation details afterward.

The 0-to-1 SWE Interview Playbook (https://www.amazon.com/dp/B0H256Z1MF?tag=sirjohnnymai-20) covers this exact question with a full interviewer-perspective rubric, showing which trade-off statements actually move the needle at each seniority level.

FAQ

Q: Which rate limiting algorithm should I default to in an interview if not told otherwise? A: Token bucket for user-facing burst tolerance, or sliding window counter if the interviewer emphasizes strict fairness; naming both and explaining the trade-off is stronger than committing to one immediately.

Q: How do I handle rate limiting across multiple gateway regions? A: Either centralize state in a globally-replicated low-latency store (accepting some replication lag as approximate limiting) or accept per-region limits summed loosely, explicitly stating that perfect global accuracy trades off against latency.

Q: What HTTP status and headers should a rate-limited response include? A: 429 Too Many Requests, with Retry-After and typically X-RateLimit-Limit/Remaining/Reset headers so well-behaved clients can back off correctly; interviewers view mentioning this unprompted as a signal of production experience.