Design a Chat System: Architecture and Data Model

Once you’ve established the high-level shape of a chat system in an interview, the follow-up conversation almost always moves into architecture and data model specifics: which components sit where, and which database actually stores the messages. This is where interviewers separate candidates who can draw boxes and arrows from candidates who understand why each component exists and what happens when it fails. This guide walks through the architecture and data model layer with the depth a mid-to-senior interview loop expects.

Core Concepts

Interview Answer Framework

Walk through architecture and data model in this order so your reasoning builds logically:

1. Lay out the component architecture end to end. Client connects to a connection gateway over WebSocket. The gateway authenticates the session and registers it in a shared session registry (commonly Redis) keyed by user ID, mapping to the specific gateway instance. When a message arrives, the gateway publishes it to a message broker, which durably logs it before any delivery attempt. A delivery service consumes from the broker, looks up the recipient’s active gateway connection(s) via the session registry, and pushes the message; if the recipient is offline, the message is persisted for later delivery.
2. Justify the database choice for message storage. Present Cassandra (or a similar wide-column store) as the default choice for message history because chat workloads are write-heavy, append-only, and naturally partition by conversation ID — Cassandra’s partition-based model scales horizontally for this access pattern without the write bottleneck a single relational primary would hit. Contrast with MySQL: relational stores are the better fit when you need strong transactional guarantees across related entities (e.g., billing, user account state) but become a scaling bottleneck for high-volume append-only message writes unless heavily sharded, which reintroduces much of the operational complexity Cassandra handles natively.
3. Design the schema around the query pattern, not the entity model. For message storage, partition by conversation ID and cluster by message timestamp (or a monotonic sequence number) descending, so “fetch the last N messages in this conversation” is a single efficient partition scan rather than a cross-partition query. Keep a separate, smaller table for conversation metadata (participants, last-message pointer) that’s read far more cheaply and updated far less often than the message table itself.
4. Tie the fan-out strategy to the data model. For fan-out-on-write, each recipient gets their own copy of the message reference written to their personal inbox partition — fast reads, but write cost scales with group size. For fan-out-on-read, a single message is written once to the conversation partition and every recipient’s client queries that shared partition directly — cheap writes, but every read has to filter for what’s new since the client’s last sync. State explicitly that you’d use fan-out-on-write for small/medium groups and fan-out-on-read (or a hybrid) for very large broadcast-style channels where writing N copies per message would be prohibitively expensive.

Common Follow-ups

Expect: “Why not just use MySQL for everything and shard it yourself?” (answer: you can, but you’re re-implementing partition management, replication, and horizontal scaling that Cassandra gives you natively for this exact write-heavy, partition-friendly access pattern), “How do you avoid hot partitions for a conversation with millions of messages, like a huge broadcast channel?” (answer: bucket the partition key by time window, e.g., conversation ID + day, so no single partition grows unbounded), and “How does the message broker guarantee ordering?” (answer: partition the broker topic by conversation ID so all messages for a given conversation land on the same partition and are consumed in write order).

Production Considerations

In production, the connection gateway is usually the first component to need careful capacity planning, since each instance holds a bounded number of concurrent connections and needs sticky session awareness at the load balancer. The message broker needs retention policies tuned to your offline-delivery SLA — if a user can be offline for up to 30 days and still expect message delivery, your broker retention or fallback durable store needs to cover that window. On the data model side, watch for the common mistake of modeling message storage around the entity relationship (users, conversations, messages as separate normalized tables joined at read time) rather than the access pattern (fetch recent messages for one conversation) — the former looks clean on a whiteboard but performs poorly at chat-scale read volume.

FAQ

Is Cassandra always the right answer for message storage? It’s the right default for high write-throughput, append-only chat history. If the interviewer’s constraints change — for example, strong consistency requirements across conversation state and billing — be ready to discuss a relational store or a hybrid model instead.

How much schema detail should I actually write out in an interview? Enough to show partition key and clustering key choices and why they match the query pattern. You don’t need full DDL, but you should be able to say “partition by conversation_id, cluster by timestamp descending” and explain what query that optimizes for.

What’s the most common architecture mistake in this interview? Treating the connection gateway as stateless like the rest of the system. It has to be stateful (or backed by a shared session registry) because a specific server instance holds a specific client’s live connection.

The most comprehensive preparation system we have reviewed for this topic is The 0-to-1 SWE Interview Playbook (Amazon: https://www.amazon.com/dp/B0H256Z1MF?tag=sirjohnnymai-20).