Real-Time Updates

The problem: the server can't speak first

Ben sends Ana a chat message, and it reaches your server. Now the server has a message for Ana and no way to hand it to her.

HTTP is ask and answer: the app sends a request, the server sends back a response. A server never sends a response nobody asked for, and it can't call Ana's phone either: phones sit behind their carrier's network with no address the internet can reach, and a browser tab can't accept requests at all. Every real-time technique is a way around that rule.

One small timeline runs through each of them below: Ana's app is open from 0 s, and Ben's message reaches the server at 7 s. When does Ana's app get it, and how many requests does it send?

Polling: ask every few seconds

The simplest fix is polling: the app asks "any news?" on a timer, say every 5 seconds, and the server answers at once, with news or empty.

time   polling every 5 s
0 s    Ana's app asks: empty answer
5 s    asks again: empty answer
7 s    Ben's message reaches the server and waits there
10 s   asks again: the answer carries the message, 3 s late
15 s   asks again: empty answer

A message that arrives just after a poll waits almost the whole interval, so polling every 5 seconds means up to 5 seconds late. And the app asks 12 times a minute whether anything happened or not. With 1 million apps open, that's 1 million ÷ 5 = 200,000 requests a second. If each person gets about one message a minute, only about 17,000 of those answers carry news: 11 polls in 12 come back empty. A shorter interval cuts the delay and multiplies the load:

poll every   late by up to   requests a second (1 million apps)
1 s          1 s             1,000,000
5 s          5 s             200,000
30 s         30 s            about 33,000

Polling is still right when news is rare and a little delay is fine: it works with any server and cache, and nothing stays open.

Long polling: hold the question

In long polling, the server doesn't answer "nothing" at once. It holds the request until there's news, or until a limit passes, say 30 seconds. The app sends a new request as soon as each answer arrives.

time   long polling, held up to 30 s
0 s    Ana's app asks; the server has nothing and holds the request
7 s    Ben's message arrives: the held request is answered with it
7 s    the app asks again at once; the server holds the new request
37 s   nothing new for 30 s: the server answers "nothing new", the app asks again

News arrives the moment it reaches the server, and there's one request per message, or one per 30 seconds when it's quiet: 2 a minute instead of 12. With one message a minute, 1 million apps send about 33,000 requests a second, a sixth of polling's load. The costs: a waiting request on the server for every open app, and a fresh request, headers and all, after every message.

WebSockets: one connection, both ways

A WebSocket starts as one HTTP request with an Upgrade: websocket header. The server answers 101 Switching Protocols, and the connection stays open, carrying small messages both ways whenever either side has something to send. At 7 s the server sends Ben's message down Ana's connection at once; at 12 s Ana replies on the same connection.

If they chat for an hour, 100 messages each way, Ana's app sends one request: the one that opened the WebSocket. No empty answers, no repeated headers. The price is on the server: every online user holds a connection open all the time, even when nobody types.

Server-sent events: a one-way stream

With server-sent events (SSE), the app sends one ordinary request, such as GET /events, and the server's response never ends: it writes each event into it as it happens. Ben's message at 7 s arrives at once.

The stream flows one way, server to app. Ana's reply goes in a separate, ordinary request, such as POST /messages. That suits apps that mostly listen: live scores, prices, an AI answer shown word by word. It's plain HTTP, so ordinary servers and proxies pass it, and the browser reconnects by itself if the stream drops.

The four side by side

way            message arrives    requests in 20 s    direction
polling 5 s    10 s (3 s late)    4                   the app asks
long polling   7 s                2                   the app asks
WebSocket      7 s                1                   both ways
SSE            7 s                1 + 1 per reply     server to app

The cost of open connections

An idle connection costs a little memory and almost no work, so connection servers are built to hold many: say about 100,000 each. Then 10 million users online at once need 10 million ÷ 100,000 = 100 connection servers, plus a few spares. Counting them like app servers, at about 1,000 requests a second, would give 10,000: an idle connection isn't a request.

• Load balancing: the balancer picks a server for each new connection and keeps the WebSocket there for its whole life: it's one long connection, not separate requests.
• Finding the user's server: Ben is on server 1 and Ana on server 2, so his message must reach the one server of 100 that holds hers. A common design gives each user a pub/sub channel (a topic of their own, in the Pub/Sub and Streams lesson's words): when Ana connects, server 2 subscribes to hers, and the chat service publishes her messages there. Another keeps a lookup, Ana → server 2. Sending every message to all 100 servers works, but 99 throw it away.
• Restarts: when a connection server dies, its connections drop. The apps reconnect to other servers after a short random wait, so they don't all arrive at once.

Which one to choose

• WebSocket: news both ways, at once: chat, multiplayer games, cursors in a shared document.
• Server-sent events (or a WebSocket): news from the server only: live scores for millions of viewers, an AI answer streamed as it's written.
• Polling every 30–60 s: rare news that can wait, like an order's status that changes a few times an hour.
• A plain request: a daily report, fetched when someone opens it.

A closed app can't hold any of these. Its notifications go through the phone's push service, Apple's or Google's.

Where it shows up in design interviews

Chat, live comments, ride tracking and collaborative editing all need live updates. Name the transport and say why: "messages go both ways and must arrive at once, so WebSockets." Then expect the follow-up: how does a message find the server holding the user's connection? Walk the path: Ben's server passes the message to the chat service, which stores it and publishes it to Ana's channel, and server 2 sends it down Ana's connection. Store first: if Ana is offline, nobody is subscribed, and the message waits for her next connection, with a push notification to her phone.

Common interview questions

Why not just poll?

For rare updates you should: it's simple and works with any server. For chat it's late and wasteful: up to 5 seconds late, and 200,000 mostly empty requests a second for 1 million apps.

WebSockets or server-sent events?

WebSockets when both sides send often, as in chat. Server-sent events when only the server has news: it's plain HTTP and reconnects by itself.

How does a message reach a user connected to another server?

The server holding the user subscribes to that user's pub/sub channel, and the chat service publishes each message there. Or keep a lookup of which server holds each user and send it straight there.

Summary

• HTTP is ask and answer: the server can't speak first.
• Polling asks on a timer: up to one interval late, mostly empty answers.
• Long polling holds the request until there's news: one request per message or per 30 s.
• A WebSocket is one open connection, both ways, with a connection per online user.
• Server-sent events stream from server to app over one request.
• Open connections need servers built to hold many, each WebSocket kept on its server, and a pub/sub channel per user to find it.