Networking Basics

The problem: packets get lost

Everything a server sends crosses the internet as small packets, each routed on its own. A busy router drops packets it has no room for, and it tells no one. Routers pass packets on and forget them, so only the two ends can fix a loss: the receiver notices the gap, and the sender, the only one with a copy, sends the packet again.

The internet's two ways of sending, TCP and UDP, make opposite choices about that. One small run shows both: a file of 5 packets, packet 3 dropped.

TCP: every byte, in order

TCP first sets up a connection with a handshake that takes one round trip. Then it numbers the data in every packet. The receiver confirms what it has ("everything up to 2"), the sender resends what was never confirmed, and the app reads one stream of bytes, complete and in order, like a file.

what happens        TCP: the app has    UDP: the app has
1 and 2 arrive      1 2                 1 2
3 is dropped        1 2                 1 2
4 and 5 arrive      1 2 (4, 5 held)     1 2 4 5
3 is sent again     1 2 3 4 5           1 2 4 5 (3 never)

When 4 and 5 arrive on time, TCP can't hand them over before 3, so it holds them. The server learns that 3 is missing only from what the laptop tells it, and sends it again a round trip or more after the first try. Then 3, 4 and 5 reach the app together.

That's the price of TCP: delay, and head-of-line blocking, where one lost packet holds up everything behind it. For a file or a bank transfer that's the right deal: a short wait beats a hole.

UDP: take what arrives

UDP sends each packet on its own: no connection, no handshake, no resend, no order. The app gets whatever arrives, as it arrives, and deals with losses itself. In the run, the UDP app has 1, 2, 4 and 5 the moment they land, and 3 never comes.

That's what you want when late data is useless: in a live video call, showing 4 and 5 now, with a small glitch, beats freezing for a moment that's already over.

• TCP: bank transfers, file downloads, chat messages, web pages and APIs.
• UDP: live voice and video calls, a multiplayer game's player positions (the next update replaces a lost one) and DNS lookups (one small question: if it's lost, ask again).

HTTP/1.1, HTTP/2 and HTTP/3

HTTP/1.1 sends one request at a time on a connection and waits for its answer, so browsers open about 6 connections to each site. A page with 30 files then takes 5 round trips even on open connections: 750 ms when the server is 150 ms away. Opening 30 connections isn't the fix: each pays its own handshakes.

HTTP/2 sends many requests at once on one connection: each request gets its own stream, and the answers come back interleaved (multiplexed). The 30 files are asked for in one round trip, about 150 ms. But HTTP/2 still runs on TCP, which sees one stream of bytes, not files: if one packet of red.jpg is lost, every packet behind it waits, whatever file it belongs to.

HTTP/3 runs over QUIC, a transport built on UDP that resends each stream on its own, so a lost packet holds up only its own file. QUIC also sets up the connection and the encryption together, in one round trip.

version    requests on a connection        the page's files
HTTP/1.1   one at a time, 6 connections    5 round trips, 750 ms
HTTP/2     all at once, on TCP             1 round trip, 150 ms
HTTP/3     all at once, on QUIC (UDP)      1 round trip; a loss holds up only its file

TLS: encryption and a certificate

TLS is the s in https. It encrypts the connection, so nobody on the way can read or change it, and proves the server is the site you asked for with a certificate for its name, signed by an authority your browser trusts.

TLS 1.3 needs one round trip to set up, after TCP's own handshake, so 2 round trips pass before the browser can send the first byte of its request:

round trip 1   TCP handshake           0 to 150 ms
round trip 2   TLS 1.3 handshake       150 to 300 ms
round trip 3   the request, the HTML   300 to 450 ms

That's 300 ms of setup across an ocean. The older TLS 1.2 needed 2 round trips of its own, 3 in all (450 ms); QUIC does both handshakes in 1 (150 ms). Inside one data center a round trip is about 0.5 ms, so the same 2 cost about 1 ms, plus the encryption work.

Where TLS ends: the load balancer

With servers behind a load balancer, the usual choice is to end TLS at the balancer: TLS termination. One place holds the certificate and does the encryption work, and the balancer can read each request's path and headers to route it, cache it or compress the reply. Behind it, requests travel on your private network, or are encrypted again when security rules require it.

Passing the encrypted bytes through to each server also works, but it blinds the balancer and puts the private key on every server. And TLS protects data on its way; stored data needs its own tool, encryption at rest.

Proxies: forward and reverse

A proxy passes requests on for someone else; which side it works for is the difference:

• A forward proxy sits in front of clients, like an office's. It can block sites, record which sites are visited, and hide which machine sent each request.
• A reverse proxy sits in front of servers, and clients never see past it. It spreads requests over the servers (load balancing), ends HTTPS, answers popular requests from its own copies and compresses replies. A load balancer is one.

An API gateway is a reverse proxy with API features: it checks API keys, applies rate limits and routes /orders and /users to their own services.

Where it shows up in design interviews

Interviewers expect you to name the transport where it matters, with a reason. Chat goes over WebSockets on TCP: every message must arrive, in order, and one long-lived connection lets the server push. Live voice, video and game state go over UDP, because late data is useless. HTTPS ends at the load balancer. And since a new HTTPS connection costs 2 round trips before the first byte, clients and services keep connections open (keep-alive, connection pools) instead of opening one per request.

Common interview questions

When would you choose UDP over TCP?

When late data is useless or cheap to ask for again: live calls, game positions, DNS lookups. TCP's resends and ordering would add delay for nothing; the app handles a loss itself, by hiding the glitch or using the next update.

What's the difference between HTTP/2 and HTTP/3?

Both send many requests at once on one connection. HTTP/2 runs on TCP, so one lost packet holds up every stream; HTTP/3 runs on QUIC over UDP, where a loss holds up only its own stream, and the connection and encryption are set up together in one round trip.

Why terminate TLS at the load balancer?

The certificate and the encryption work live in one place, and the balancer can read requests to route, cache and compress them.

Why reuse connections?

Every new HTTPS connection pays TCP's and TLS's handshakes, 2 round trips, before the first request: 300 ms across an ocean. Keep-alive and connection pools pay that once.

Summary

• TCP: a connection (1 round trip to set up), every byte in order; a lost packet is resent and holds up everything behind it.
• UDP: single packets, no handshake, no resend, no order; for calls, games and DNS.
• HTTP/1.1 sends one request at a time per connection (about 6 connections per site), HTTP/2 many at once on one TCP connection, HTTP/3 the same over QUIC on UDP.
• TLS encrypts and proves the server's identity with a certificate; with TCP it costs 2 round trips (300 ms across an ocean) before the first byte.
• End TLS at the load balancer, a reverse proxy; forward proxies work for clients.