Video Calls (Zoom)

The problem

"Design video calls like Zoom." Up to 10 people see and hear each other live, from laptops on fast home internet and phones on trains. In a conversation, every delay shows.

It reuses Networking Basics (live media over UDP) and Real-Time Updates (events like "Ben joined" pushed over a WebSocket). New here: who sends video to whom in a group, a quality for each viewer, signaling apart from the media, and recording. Park webinars for thousands of viewers (live streaming) and text chat: each is a design of its own.

The numbers

From the interviewer: calls of up to 10 people, everyone's video on. Say one video stream takes about 1 Mbps (a megabit a second) and a home connection uploads about 5 Mbps.

setup              streams in the call   each person uploads
peer to peer       10 × 9 = 90           9 streams, 9 Mbps
through a server   10 up, 90 down        1 stream, 1 Mbps
with simulcast     10 up, 90 down        3 sizes, 1.4 Mbps

Peer to peer asks each home connection for almost twice what it can upload. Through a server, the server does the copying: a call of 10 takes in 10 Mbps and sends out up to 90. Say one server's network sends about 1 GB a second, 8,000 Mbps: close to 90 such calls at full size, more when viewers take small copies.

Group calls: a forwarding server

With 2 people, the apps can send their video straight to each other: one upload each, no server in between, the lowest delay. With 10, going direct means 9 uploads each, 9 Mbps, and a phone encoding 9 streams at once.

Instead, each app uploads its video once to a media server that forwards it to the other 9. The common kind is a Selective Forwarding Unit (SFU): it copies packets to the right people without decoding them, so it's cheap and adds little delay. A mixing server (an MCU) instead decodes all 10 videos and encodes one combined picture: one stream down for each person, but heavy work on every call, more delay and a fixed layout. Many apps go peer to peer for 2 people and move the call to a media server when a third joins.

Peer to peer must also get through home routers, which hide the devices behind them (NAT): apps try a direct path first and fall back to a relay server that passes the packets on (TURN) when it fails.

A quality per viewer: simulcast

Ben joins from a phone on a train that downloads about 1 Mbps, and he needs 9 videos. Nine full copies would be 9 Mbps, 9 times what he can take. Sizing everyone's call for one weak link blurs it for all, and shrinking videos on the server brings back the mixing cost.

With simulcast, every app uploads 3 sizes of its video, say 1, 0.3 and 0.1 Mbps, 1.4 Mbps in all, still under the 5 Mbps upload. The server forwards each viewer the copy that fits: Ben gets 9 small ones, 0.9 Mbps, and a fast laptop gets the big copy of the speaker and medium ones for small tiles. Choosing a copy is just choosing which packets to forward, so when Ben's train leaves the tunnel, the server switches him to bigger copies.

Signaling and media

Signaling sets the call up and keeps everyone in step: who joins and leaves, who's muted, which media server hosts the call, which formats and sizes each app can send. These messages are small and rare, and every one must arrive in order, so they travel over a WebSocket on TCP to a signaling service, as in Real-Time Updates:

→ {"type": "join", "meeting": "standup"}
← {"type": "joined", "server": "media.eu.example.net", "people": ["ana", "ben"]}
← {"type": "muted", "by": "ben"}

The media is big and steady, and a late packet is useless: about 50 packets of voice a second, each about 20 ms of sound, plus the video frames. It goes over UDP straight to the media server, never through signaling, so a busy signaling service can't delay anyone's voice. Where a firewall blocks UDP altogether, the media can fall back to TCP through a relay, with TCP's delays.

Recording

The interviewer wants recordings in the cloud, ready even if everyone has left. When the host presses record, the signaling service starts a recorder on the server side that joins the call as a hidden participant. It receives every stream from the media server like any viewer, mixes them into one video, and writes the file to object storage, with its link in the meetings database. Mixing is heavy, but it happens once per recorded call, not for every viewer.

An hour at about 1 Mbps is about 450 MB. Recording on the host's laptop depends on one machine that might close or crash, and the file takes about 12 minutes to upload at 5 Mbps. Saving the forwarded packets in a database gives 10 separate streams in 3 sizes each, not a video anyone can play.

Same pattern as Networking Basics: live media over UDP

Networking Basics' core carries over: voice and video ride UDP, and the app hides a lost packet instead of waiting, since a short glitch beats a freeze. Real-Time Updates adds the WebSocket for signaling. What's new is the shape of the call: who sends to whom, at which size, and who records.

Summary

• Scope: calls of up to 10, a quality per viewer, recording.
• Data: meetings and who's in them; recordings in object storage.
• API: signaling over a WebSocket: join, leave, mute, which server.
• Numbers: 10 people direct is 90 streams, 9 Mbps up each.
• Scale: an SFU forwards: 1 Mbps up each; peer to peer for 2.
• Quality: simulcast, 3 sizes up, the copy that fits each viewer down.
• Recording: a hidden participant mixes the call into object storage.