Twitch

The problem

"Design Twitch." A streamer plays a game live while thousands watch and chat, and she reads the chat and answers as she plays. Viewers pick a quality that fits their connection, and a popular streamer's audience arrives within minutes of going live.

This practice problem reuses the pattern of Design YouTube: several qualities (renditions), each cut into segments, a player that picks each segment's quality from its measured speed, and CDN edges near the viewers. The chat beside the video is Design Live Comments: connection servers, a pub/sub channel per stream, and only some messages shown when a big chat floods. What's new is that the video is made while it's watched. Park watching past streams: that's YouTube again, on the saved segments.

The numbers

From the interviewer: a popular streamer has 100,000 viewers, each watching about 4 Mbps of video on average, and she sends one 6 Mbps stream from home.

what        working                  result
video out   100,000 × 4 Mbps         400 Gbps
in bytes    400 Gbps ÷ 8 bits        50 GB a second
servers     50 GB ÷ 1 GB a second    about 50 servers' worth
coming in   her one 6 Mbps stream    0.75 MB a second

The common slip is reading 400 gigabits as 400 gigabytes. The other is stopping at what comes in: her home connection sends one stream, and the system has to make the 100,000 copies.

Qualities: transcoded in real time

Her streaming software sends one 1080p stream, commonly over RTMP, to an ingest server near her. Many viewers are on phones that need 480p or less. YouTube transcodes a finished file off a queue, but here there's no finished file, so transcoders beside the ingest server turn each few seconds of 1080p into lower qualities as it arrives:

quality   bitrate
1080p     6 Mbps (her stream)
720p      3 Mbps
480p      1.5 Mbps
360p      0.8 Mbps

They must keep up: each second of video has to be transcoded in under a second, or the stream falls further behind. Waiting for the stream to end would leave live viewers only her 1080p, and a phone on 2 Mbps can't play 6 Mbps.

Segments: delay against smoothness

Each quality is cut into segments listed in a playlist, as in YouTube. A player keeps about 3 segments in hand, downloaded but not yet played, so a slow moment doesn't stop the video; say transcoding and delivery add about 1 second. The segment length sets the rest:

segment   behind live             in hand   requests, 100,000 viewers
10 s      3 × 10 + 1 = 31 s       30 s      10,000 a second
2 s       3 × 2 + 1 = 7 s         6 s       50,000 a second
0.5 s     3 × 0.5 + 1 = 2.5 s     1.5 s     200,000 a second

Ten-second segments, as in the YouTube design, are smooth but put viewers 31 seconds behind: a viewer who asks her something in chat sees her answer half a minute later, while chat arrives in about a second. Half-second segments cut the delay but bring 4 times the requests, each with its own overhead, and only 1.5 seconds in hand, so any hiccup stalls the video. Two seconds is a common middle for live: about 7 seconds behind, enough in hand, and 50,000 requests a second that a CDN handles easily. Low-latency modes of HLS and DASH go further by sending parts of a segment while it's still being made.

The live path

1. Her streaming app sends the 1080p video to the ingest server.
2. Transcoders make 720p, 480p and 360p of it as it arrives.
3. Each quality's new 2-second segment is saved at the origin.
4. Each quality's playlist adds the new segment's name.
5. A viewer's player reads the playlist and asks a CDN edge for the segment.
6. The player plays it, about 7 seconds after her camera saw it.

It runs every 2 seconds. Save before listing: a playlist that names a segment before it's saved sends players after something that isn't there.

Popular streams: edges warm up

At 8 pm she goes live, and 100,000 viewers arrive within minutes, spread over 100 CDN edges (say). Each new segment exists only seconds before they all want it. A YouTube premiere can be pushed to the edges in the quiet hours before; a live segment can't, because there is no before.

Instead each edge fetches each new segment from the origin once, on its first request, and hands that copy to every viewer near it; requests that arrive during the fetch wait for it. For each quality the origin sends each segment about 100 times instead of 100,000. An edge with no viewers of her stream fetches nothing, and warms up as viewers near it arrive. Passing every request on to the origin would mean 50 GB a second from one place, as if there were no CDN.

Same pattern as YouTube and Live Comments

From YouTube: renditions cut into segments, a playlist (the manifest), adaptive streaming that picks each segment's quality from the measured speed, and CDN edges near the viewers. From Live Comments: the chat, on connection servers with a pub/sub channel per stream. What changes is time. Nothing exists ahead, so transcoding happens as the stream arrives, segments are short, and edges fill as viewers come.

Summary

• Scope: watch live, seconds behind, in a quality that fits; chat as in Live Comments.
• Numbers: about 50 GB a second out to 100,000 viewers, from one 6 Mbps stream in.
• Services: ingest servers and real-time transcoders; an origin.
• Segments: 2 seconds: about 7 s behind live, 50,000 requests a second.
• Live path: save the segment, then list it; players keep 3 segments in hand.
• Popular streams: each edge fetches each segment once and warms up as viewers arrive.