Video View Counts

The problem

"Design the view counts of a video site." Every video shows its views, and creators are paid by them. The interviewer adds three rules: a view counts only after 30 seconds of watching, one person's replays count at most once per video per hour, and payouts must be exact.

This practice problem reuses the pattern of Design a Distributed Counter. What changes is that a view now has to be earned and checked, because money hangs on it: the counter's "a few off is fine" doesn't hold for a payout. Park watch time and comments.

The numbers

From the interviewer: 1 billion counted views a day, as in the Distributed Counter, with busy hours at 5× the average: about 60,000 views a second (1 billion ÷ 86,400 × 5). Counting once per hour means remembering each (viewer, video) already counted this hour, about 50 bytes each:

• An average hour: 1 billion ÷ 24 ≈ 40 million views, about 2 GB of pairs.
• A busy hour: 5× that, about 200 million pairs, about 10 GB.
• A whole day's pairs would be 50 GB, but a pair matters only within its hour.

A real view: a signed play token

The player runs on viewers' devices, so the server can't take its word. When a play starts, the server hands the player a play token holding the video, the viewer and the server's own start time, signed with a key only the server has. At 30 seconds the player sends it back:

POST /videos/v42/plays
→ 201 Created   {"token": "Qmx…Zr"}

POST /videos/v42/views   {"token": "Qmx…Zr"}
→ 202 Accepted

The server checks the signature and that at least 30 seconds have passed since the start time inside. A forged time breaks the signature, and a report sent early fails the check. Counting at the start would count visitors who leave after 2 seconds; trusting the player would let a script send POST /view a million times. The token proves that time passed, not that anyone watched: a script that waits still gets through, which is what the next checks are for.

Repeats: once per viewer per video per hour

Every report checks its (viewer, video, hour) pair, at least 60,000 checks a second at busy hours. Keep the pairs in memory: set each key only if it's absent, in one step, expiring when its hour ends, and count the view only if the set worked. One step matters: if the check and the save were separate, two reports of the same play arriving together would both find nothing and both count. About 10 GB fits a few cache servers.

Ana plays v42   key                 counted?
10:05           (ana, v42, 10:00)   yes
10:20           (ana, v42, 10:00)   no, a repeat
10:50           (ana, v42, 10:00)   no, a repeat
11:10           (ana, v42, 11:00)   yes
13:30           (ana, v42, 13:00)   yes

Clock hours are simple: plays at 10:59 and 11:01 both count, which is fine for a rule meant to stop 100 replays. A views table would cost a database read and write per report, 60,000 a second where one server does about 5,000, on a table growing by a billion rows a day. A list in the player means nothing to a bot.

The public count: pause while checking

Tokens and repeat keys judge one view at a time, and a bot farm can pass both. So a view checker reads the stream before the counts: it passes ordinary views on and holds suspicious bursts, such as v42 jumping from about 100 views a minute to 5,000, mostly from one network. While the burst is held, v42's count pauses; once it's checked, the real views are added and the count catches up. Some sites show a count that pauses while views are checked. The price is a public count a few minutes behind, which nobody minds.

Adding everything at once and subtracting fakes later would make counts fall, and show cheaters which views were caught. Hiding every count until the nightly recount would leave every video a day behind.

Payouts: a nightly recount

Creators are paid by their views, say every month, so a payout can't come from the live count. Some fraud only shows across a whole day, like thousands of new accounts all watching the same few videos, and some is found later still. Each night a recount job re-reads the day's views (the stream keeps a few days; object storage keeps a copy for audits), drops repeats by token, applies the latest rules, corrects the totals and records each creator's views for the payout. It's the Ad Click Aggregator's reconciliation: advertisers are billed from a recount of the raw clicks, not from the streaming counts.

check                                    when
token real, 30 seconds passed            as views stream in
already counted this hour                as views stream in
a sudden burst from one network          as views stream in (held)
thousands of new accounts, same videos   nightly recount
devices found later to be bots           nightly recount
each creator's views for the payout      nightly recount

Same pattern as the Distributed Counter

Everything after the checks carries over. Each video's count sits on its row behind a cache kept about a second; a viral video's row is split into sharded counters; counter servers add up views in memory and write once a second per video. A crash that replays part of the stream may add a few views twice to the public count, which is fine there; the recount keeps payouts exact.

Summary

• Scope: views after 30 seconds, replays once per hour, exact payouts.
• Numbers: about 60,000 views a second; about 10 GB of pairs in a busy hour.
• Real views: a play token signed with the server's own start time.
• Repeats: set-if-absent keys in memory, expiring with their hour.
• Public count: a few minutes behind, pausing while a burst is checked.
• Payouts: a nightly recount of the checked views.