System Design: Chat Application Architecture

“Design a chat application” is one of the most frequently asked system design interview prompts in 2026, appearing across FAANG, mid-size, and startup loops alike. It’s popular precisely because it forces candidates to reason about real-time delivery, consistency, storage scaling, and presence — all in a 45-minute window. This guide breaks down a production-grade architecture and the decision points interviewers actually probe.

Requirements Clarification

Before architecture, scope the problem. A strong candidate opens by clarifying:

• 1:1 messaging vs group chat — group chat introduces fan-out complexity
• Message delivery guarantees — at-least-once vs exactly-once
• Online presence and typing indicators — real-time state beyond messages
• Media support — images/video change storage architecture significantly
• Scale target — 2026 reference numbers: 50M DAU, 500M messages/day, peak 50K messages/second

Skipping this step is the single most common reason candidates get marked down — jumping straight to “I’ll use WebSockets and Kafka” without establishing why.

Core Architecture Components

A chat system decomposes into five subsystems:

1. Connection layer (WebSocket gateway) Clients hold persistent WebSocket connections to a gateway tier. Because a single server can hold roughly 50K-100K concurrent WebSocket connections (bounded by file descriptors and memory), you need a connection registry — typically Redis — mapping user_id → gateway_server_id so any backend service knows which gateway node holds a given user’s live socket.

2. Message service Receives a message, validates it, persists it, and triggers delivery. This is the write path and must be idempotent — clients retry on network blips, so dedupe on a client-generated message_id.

3. Storage layer Messages are append-only and read by recency, making them a canonical write-heavy, time-ordered dataset. Cassandra or DynamoDB (partitioned by conversation_id, clustered by timestamp) are the standard 2026 choices over a relational database — you rarely need cross-conversation joins, and horizontal write scaling matters more than transactional consistency here.

4. Fan-out / delivery service For 1:1 chat, fan-out is trivial (one recipient). For group chat with N members, you choose between:

• Fan-out on write: push the message to all N members’ queues immediately. Fast reads, expensive writes for large groups.
• Fan-out on read: store once, each client pulls on reconnect/poll. Cheap writes, more complex read path.

Most production systems (WhatsApp, Slack) use fan-out on write for groups under ~500 members and a hybrid pull model for larger channels, since a write-fan-out to 10,000 members per message is prohibitively expensive.

5. Presence service Tracks online/offline/typing state. This is inherently ephemeral, high-write, low-durability-requirement data — a strong signal to use Redis with TTL-based keys rather than your durable message store.

Data Model

A minimal schema for the message store:

Table: messages
  conversation_id (partition key)
  message_id / timestamp (clustering key)
  sender_id
  content
  media_url (nullable)
  status (sent/delivered/read)

Table: conversations
  conversation_id (partition key)
  member_ids
  last_message_preview
  updated_at

Table: user_devices
  user_id (partition key)
  device_id
  gateway_server_id
  last_seen

Partitioning by conversation_id keeps a conversation’s messages co-located for fast timeline reads, but creates a hot-partition risk for extremely active group channels — mitigated by time-bucketing the partition key (e.g., conversation_id + day_bucket).

Comparison: Delivery Strategy Tradeoffs

Handling Offline Delivery and Consistency

When a recipient is offline, the gateway registry lookup misses, and the message service writes to a per-user offline queue instead of pushing live. On reconnect, the client requests all messages since its last acknowledged message_id (or timestamp checkpoint) — this is why message ordering per-conversation and idempotent client IDs matter more than global ordering across conversations.

For consistency, most interviewers expect you to reject strict consistency (unnecessary latency cost) in favor of eventual consistency with read receipts as a separate, asynchronous confirmation signal — decoupling “message stored” from “message displayed as read.”

What Interviewers Are Actually Scoring

In 2026 loops, the bar has shifted from “can you draw boxes and arrows” to “can you justify each tradeoff under a changing constraint.” Interviewers will pressure-test your design by changing a requirement mid-interview — “what if this group has 100,000 members” or “what if we need end-to-end encryption” — and watch whether your architecture bends or breaks. Candidates who pre-empt these follow-ups (e.g., mentioning the fan-out hybrid model unprompted) score meaningfully higher than those who only answer the literal prompt.

This exact skill — anticipating the follow-up before it’s asked — is one of the patterns broken down in detail in The 0-to-1 SWE Interview Playbook, which includes a full chat-app design walkthrough with the specific follow-up questions real interviewers ask at each stage.

FAQ

Q: Should I use WebSockets or Server-Sent Events (SSE) for the connection layer? A: WebSockets, because chat requires bidirectional communication (client sends messages, not just receives). SSE is one-directional (server-to-client) and better suited to notification feeds, not chat.

Q: How do you handle message ordering across multiple devices for one user? A: Assign a monotonically increasing sequence number per conversation (not globally), and have each device track its own last-synced sequence number. Global ordering across conversations is unnecessary and expensive to guarantee.

Q: What’s the biggest architectural mistake candidates make in this interview? A: Treating chat as a stateless HTTP CRUD problem and forgetting that persistent connection state (which gateway server holds which user) is itself a piece of distributed state that needs its own registry and failure handling.