Api Gateway Pattern Microservices Routing

Why The API Gateway Pattern Is A Recurring Interview Topic

Any system design interview involving microservices, and by 2026 that’s the majority of backend-focused loops at mid-size and large companies, eventually asks: “how do clients talk to your 40 microservices without every client knowing about all 40?” The answer is the API gateway pattern, and interviewers use it to probe whether you understand cross-cutting concerns (auth, rate limiting, request routing, protocol translation) versus business logic, and whether you know the newer service mesh alternative well enough to justify picking one over the other.

What The API Gateway Actually Does

An API gateway sits as a single entry point between external clients and internal microservices. It performs request routing (mapping /users/* to the user service, /orders/* to the order service), protocol translation (REST-to-gRPC, or WebSocket upgrade handling), authentication and authorization (validating JWTs or OAuth tokens once, at the edge, rather than in every service), rate limiting and throttling per client/API key, and response aggregation (the Backend-for-Frontend variant, composing multiple downstream calls into one client-facing response).

The critical interview insight: the gateway centralizes cross-cutting concerns so individual services don’t reimplement auth, logging, and rate limiting independently. This is a tradeoff, not a free lunch: you’ve now introduced a single point of coupling, and if the gateway team owns routing config, every new service launch requires a gateway change, which becomes an organizational bottleneck at scale.

Routing Strategies

Path-based routing (the most common): /api/v1/orders/* routes to the order service, /api/v1/users/* to the user service. Simple, cacheable, and what tools like Kong, AWS API Gateway, and Envoy support natively out of the box.

Header/version-based routing: route based on an Accept-Version header or a custom header, letting you run v1 and v2 of the same logical API simultaneously during a migration without changing the URL structure. This is the pattern most large companies use for backward-compatible API evolution.

Canary/weighted routing: route a percentage of traffic (5%, 10%) to a new service version based on weighted rules, critical for safe rollouts. In 2026, this is typically delegated to the service mesh’s sidecar layer (Envoy via Istio) rather than the gateway itself, since mesh-level routing operates with lower latency overhead and finer-grained control per internal hop.

Gateway Vs Service Mesh: The 2026 Answer

This is the single highest-signal follow-up in these interviews. The distinction: an API gateway handles north-south traffic (external client to internal services), while a service mesh (Istio, Linkerd, or increasingly ambient-mode meshes without sidecars) handles east-west traffic (service-to-service communication inside the cluster).

By mid-2026, the dominant production pattern for companies past ~30 services is both together: a lightweight edge gateway (Kong, Envoy Gateway, or a cloud-native option like AWS API Gateway/ALB) for external auth and rate limiting, plus a service mesh for internal mTLS, retries, circuit breaking, and fine-grained traffic shifting between services. Istio’s ambient mode (stable since 2024-2025) has meaningfully reduced the sidecar resource tax that made meshes expensive to adopt, which is why more mid-size companies now run a mesh internally even without Google/Netflix-scale traffic.

Do NOT tell an interviewer “just use a service mesh for everything,” since that signals you don’t understand the latency and operational cost of running a sidecar proxy on every pod, particularly for latency-sensitive services doing sub-10ms internal calls.

Comparison Table

Failure Modes To Raise Proactively

Two things separate a strong answer from an average one. First: the gateway is a single point of failure by design, so it must be deployed with redundancy (multiple stateless instances behind a load balancer, never a single node) and health-checked aggressively; a gateway outage takes down every downstream service simultaneously even if those services are healthy. Second: gateway-level rate limiting needs a distributed counter (Redis with a sliding window or token bucket algorithm) rather than in-memory counters per instance, or a client can trivially bypass limits by hitting different gateway replicas.

For a deeper walkthrough of how to frame this exact tradeoff discussion under interview time pressure, The 0-to-1 SWE Interview Playbook (https://www.amazon.com/dp/B0H256Z1MF?tag=sirjohnnymai-20) includes a dedicated section on microservices communication patterns and the gateway/mesh decision tree.

FAQ

Q: Do I need a service mesh if I only have 10 microservices? A: Usually not. Below roughly 15-20 services, the operational overhead of running a mesh (sidecar management, control plane upgrades) typically outweighs the benefit; a simple gateway plus service-to-service auth via mutual TLS certs managed manually or via a lightweight library is often sufficient.

Q: What’s the biggest mistake candidates make when designing an API gateway? A: Treating it as stateless when it isn’t, forgetting that rate limiting and session-affinity logic need a shared, distributed backing store (Redis), not per-instance memory, which breaks the moment you scale the gateway horizontally.

Q: Can the API gateway also do response caching? A: Yes, and it’s a strong point to raise unprompted: gateways commonly cache idempotent GET responses at the edge (with cache-control headers or an explicit TTL policy), reducing load on downstream services for high-traffic read paths without touching service code.