Chat (WhatsApp)

What the interviewer is asking

"Design WhatsApp" is one of the most common system design questions. Storing text is the easy part. The hard part is delivery: a message has to reach a phone that may be connected to any one of a thousand servers, or switched off, while the sender watches the ticks change from ✓ to ✓✓.

The walkthrough below follows the 8 steps of the interview framework: scope, data, API, the busy path, numbers, services, scale and deep dives.

1. Scope: what version 1 must do

• Send a text message to one person, or to a group of up to 100.
• Deliver it at once if they're online, in well under a second, or later if their phone is off: a message is never lost.
• Ticks: ✓ when the server has the message, ✓✓ when it reaches the other phone.

Park the rest out loud: photos, videos and voice notes, voice and video calls, message search, and one account on several devices. Blue "read" ticks are parked too: they'd be one more receipt, sent when the chat is opened, along the same path as ✓✓.

Mention security in one sentence: WhatsApp encrypts messages end to end, so its servers pass on text they can't read. The server still knows who wrote to whom and when, but it can't search or scan the text.

2. Data: messages, conversations and members

record         fields
message        id, conversation_id, sender_id, text, time
conversation   id, kind (one-to-one or group), name
member         conversation_id, user_id

A one-to-one chat is a conversation with two members, so groups need no extra tables. History is read by conversation, newest first, 50 messages at a time.

Message ids should sort by time. Snowflake-style ids (time, then machine, then a counter, from the Unique IDs lesson) are made on any chat server without a shared counter, and a larger id is a later message, so "the 50 before m101" is just the ids below m101. Random UUIDs lose that order, and phone clocks are often off by minutes, so a reply could sort before the message it answers. For one exact order, the chat service also gives each message a sequence number in its conversation (1, 2, 3…), so both phones show the same order.

A note on WhatsApp itself: its privacy policy says messages live on people's phones and are deleted from its servers once delivered (undelivered ones wait, encrypted, for up to 30 days). Keeping history on the server, as here, is the usual interview answer; say which you choose.

3. The API: a WebSocket and plain HTTP

Messages and receipts can arrive at any moment, so each online phone keeps one WebSocket open (the Real-Time Updates lesson compares it with polling). Both sides send small events on it; → is from the phone, ← from the server:

GET /ws   Upgrade: websocket   → 101 Switching Protocols, and the connection stays open
→ {"type": "send", "chat": "c42", "text": "Running late"}
← {"type": "message", "id": "m101", "chat": "c42", "from": "ana", "text": "Running late"}
← {"type": "receipt", "id": "m101", "status": "delivered"}

Whatever the phone asks for itself is an ordinary HTTP request:

GET  /conversations/c42/messages?before=m101   → 200 OK  the 50 messages before m101
POST /conversations  {"members": ["ana", "ben", "cy"]}   → 201 Created  {"id": "c43"}

4. The busy path: a message to someone online

1. Ana's app sends "Running late" on her WebSocket.
2. The server saves it with an id, and Ana sees ✓.
3. The server sends it down Ben's open WebSocket.
4. Ben's app tells the server it got it, and Ana sees ✓✓.

Save first, then send: once the message is stored, nothing that fails later can lose it. Each hop inside the data center is a round trip of about 0.5 ms; the phones' own networks take most of the time, and the trip stays well under a second.

5. The numbers

From the interviewer: 500 million daily users, each sending about 40 messages a day, and about 100 million online at busy hours. Say a message takes about 100 bytes with its ids and time. A day has 86,400 seconds; round it to 100,000.

messages     500 million × 40 = 20 billion a day ÷ 100,000 s ≈ 200,000 a second (≈ 230,000 with 86,400)
busy hours   5 × 200,000 ≈ 1 million messages a second
storage      20 billion × 100 bytes = 2 TB a day ≈ 730 TB a year (over 2,000 TB with 3 copies)
connections  100 million online ÷ 100,000 per server = 1,000 connection servers

Two of these shape the design. Storage grows by terabytes a day, so messages are split by conversation id: each chat's history sits whole on one shard, and a "before" read is one range scan. And it takes about 1,000 servers just to hold the connections, so sender and receiver are almost never on the same one.

6. Services: connection servers and a chat service

• Connection servers hold the WebSockets, about 100,000 each, pass incoming messages on and push outgoing ones down the right connection. Nothing else.
• The chat service gives each message its id, saves it, works out who should get it and tracks receipts.

They grow differently: connection servers with people online, the chat service with messages a second. Restarting a connection server drops 100,000 connections, so keep it simple and rarely changed; the chat service holds none, so it can be updated any time.

7. Scale: finding the right connection server

Ana's WebSocket is on server 3, Ben's on server 7. Her message comes in through server 3 and has to leave through server 7, the only one that can reach Ben's phone. Asking all 1,000 servers would cost 1,000 questions per message: 200 million a second.

The fix is a session store. When a WebSocket opens, its server writes "Ben → server 7"; when it closes, the entry goes. The chat service looks Ben up and forwards the message to server 7, with a direct call or on server 7's pub/sub channel (Real-Time Updates uses a channel per user; both work). A reply runs the other way: Ben's "No problem" comes in on server 7, and the chat service looks up Ana and sends it to server 3.

It's read for every message and every receipt: about 1 million lookups a second at busy hours for messages, as many again for receipts. Keep it in memory: at about 100,000 a second per server, that's about 20 servers, split by user id.

8. Deep dive: offline messages and receipts

When Ben's phone is off, the session store has no entry for him. The chat service saves Ana's m102 (she sees ✓), marks it undelivered and puts a job on a queue that sends a push notification through Apple's or Google's push service. When Ben comes back:

1. His app opens a WebSocket (on server 12, say), and that server writes Ben in the session store.
2. His app asks for the messages it hasn't got, and the chat service sends m102.
3. His app acks m102, and the chat service marks it delivered.
4. The chat service looks up Ana and sends her the ✓✓.

The ack itself can get lost when the connection drops just after m102 arrives. The server still has m102 as undelivered and sends it again next time. Delivery is at least once, as in the Message Queues lesson, and the app makes it safe by keeping messages by id:

moment                  on the server      in Ben's chat
m102 sent, ack lost     m102 undelivered   m101 m102
reconnect, m102 again   m102 undelivered   m101 m102 (the copy is ignored)
second ack arrives      m102 delivered     m101 m102

Ana's app resends when her connection drops before ✓, so it gives each message an id first and the chat service ignores a copy. In a group, WhatsApp shows ✓✓ once every member has the message, so acks are counted per member.

9. Deep dive: groups

A group message is saved once, for the conversation. Then the chat service reads the members, looks each one up and forwards the message to their server. Family has 6 members, and Eli's phone is off:

member   session store   what happens
Ana      server 3        the sender
Ben      server 12       forwarded
Cy       server 7        forwarded
Dee      server 12       forwarded
Eli      no entry        waits, undelivered, with a push notification
Fay      server 3        forwarded

That's 5 lookups and 4 forwards (Ben and Dee share server 12, so theirs can travel together). A group of 100 makes 99 lookups per message: fine. A group of 100,000 would make 100,000, for the same reason as a celebrity's tweet on Twitter, so a common design stops pushing past a size and lets members fetch new messages when they open the group.

10. Deep dive: presence

A phone that loses signal in a tunnel sends no goodbye, so the server can't tell a quiet user from a gone one. The app sends a tiny heartbeat on its WebSocket, say every 30 seconds, and the connection server watches for silence:

time    what server 12 sees
0 s     a heartbeat: Ben is online
30 s    a heartbeat
60 s    a heartbeat
75 s    (Ben's signal is lost: the server can't know)
90 s    no heartbeat; one late beat can be a slow network
120 s   a minute of silence, 2 missed: offline, last seen 60 s

The interval is a trade-off. Every 30 seconds, 100 million phones send about 3.3 million heartbeats a second, about 3,300 per connection server, and a dead phone shows offline within a minute. Every second would be 30× more and keep every phone's radio awake, draining batteries. Every 10 minutes would leave a dead phone "online" for up to 20 minutes, its messages sent to a dead connection instead of waiting with a push notification. Only changes are written to the session store, and a change goes only to people who have that chat open.

What breaks first

• A connection server crashes: 100,000 phones reconnect at once. Apps wait a random second or two first, so the load spreads.
• Stale session entries: Ben's old server may notice his dead connection after his new one is open. It removes his entry only if the entry still names itself.
• The messages database: 2 TB a day. Shard it by conversation id.
• Push notifications: one can arrive late or not at all, so the message always waits on the server until the app fetches it.

Summary: the order to follow

1. Scope: texts to one person or a group, now or when back online, with ✓ and ✓✓.
2. Data: messages, conversations and members; Snowflake-style ids and a sequence number per conversation.
3. API: a WebSocket per online phone; GET /conversations/{id}/messages?before=… for history.
4. Busy path: a message to someone online, saved first.
5. Numbers: about 200,000 messages a second, 2 TB a day, 1,000 connection servers.
6. Services: connection servers hold WebSockets; a chat service saves and routes.
7. Scale: a session store says which server holds each user.
8. Deep dives: offline messages wait and are kept once by id; groups fan out per member; heartbeats drive presence.

The same pieces carry over to Slack, Discord or live comments: hold the connections in one place, keep the logic in another, and always know which server holds whom.