Craigslist

The problem

"Design Craigslist." People post free listings, such as a sofa, an apartment or a job, in their city and a category. Others search them, say road bikes for sale in Denver, and reply by email. A listing comes down after 30 days and can carry a few photos, and neither buyer nor seller should see the other's real email address.

This practice problem reuses the Search Engine's pattern. Park payments (people pay when they meet) and seller ratings.

The numbers

From the interviewer: 3 million new listings a day, each up for 30 days, with about 4 photos of 1 MB each, and 20,000 searches a second at busy hours. Say a listing's text and its index entries take about 2 KB.

what             working
live listings    3 million × 30 days = 90 million
text and index   90 million × 2 KB = 180 GB
photos           90 million × 4 MB = 360 TB (about 1.1 PB in 3 copies)
new listings     3 million ÷ 86,400 ≈ 35 a second

As many listings come down each day as go up, so the live set stays at a month's worth. Two slips: one day of listings (3 million) forgets that each stays 30 days, and a year (about 1.1 billion) forgets that they expire. 180 GB is more than one server keeps in memory, about 100 GB, so the listings must be split. Photos go to object storage and a CDN, as in Instagram.

Splitting by city

Nearly every search names one city, so split the listings and their index by city: a search for road bikes in Denver goes to Denver's shard alone and reads only Denver's listings.

The Search Engine split by document, because a web search covers every document anyway. Here that is waste: split by listing id over 20 shards, every search would ask all 20, about 400,000 shard searches a second at busy hours instead of 20,000. Splitting by category fails too: the busiest categories take most of the load, and a search for bikes in Denver would still read every city's for-sale listings.

Cities are uneven, so size the shards to them: New York gets its own, with copies for its searches, and many small cities share one. A rare search across a region asks a few.

Expiry reaches the index

As in Pastebin, every read checks the expiry date, and a daily job deletes what's past it. What's new is the search index, a second copy of every listing. Listing 4821 reached its 30 days this morning: its page already answers 404, but a search for sofa reads Denver's index, which still lists 4821.

listing   age       expires
4821      30 days   today
5310      12 days   in 18 days
6007      1 day     in 29 days

So each index entry also holds its listing's expiry, and searches skip expired ones, exact at day 30. The daily job removes those entries, deletes the photos, and deletes the rows last, so a crash halfway leaves rows that still point at what's left. Each CDN copy of a photo lives only as long as its listing has left; a listing removed early, when the sofa sells, needs a purge. Rebuilding each city's index nightly would reread all 90 million listings to drop 3 million, and a dead listing would still show for up to a day.

The email relay

Give each listing its own relay address, shown instead of the seller's email:

GET /listings/4821
  → 200 OK
{"title": "Road bike, 54 cm",
 "price": 250,
 "city": "denver",
 "reply_to": "k7x2m@relay.example"}

The relay keeps relay address → listing → the seller's real email. A buyer's mail reaches the seller from a relay address made for that buyer, so the reply comes back through the relay too, and neither side sees a real address. When the listing expires or sells, its address stops working. Every message passes one place, where it can be rate limited and checked against known scams.

A contact form that hides the seller works one way only: the seller's reply shows their address. One relay address per seller never stops working and ties all their listings together.

Spam and scams

Free posting draws spam. Say posting needs an account with an email address. Each abuse gets its own defense:

• A rate limit per account: a few posts a day per category and city, one ad in one city. That stops the same sofa posted 200 times a day, or in 40 cities at once.
• Phone verification: each account confirms a phone number with a code sent by text. Numbers cost money and are far scarcer than email addresses, so a script opening 1,000 accounts needs 1,000 numbers, and a ban sticks to the number.
• Flags and review: no limit can tell a real apartment from a fake one, or spot a deal that asks for a deposit first. Readers flag; enough flags hide a listing until a reviewer looks.

Same pattern as the Search Engine

A listing service writes and a search service reads, each growing with its own load. Each city's shard holds an inverted index from words to listing ids, plus a small fresh index in memory so new listings show up within minutes. A result cache takes repeated searches, such as apartments in New York. What changes is the split: one city per search, so one shard per search.

Summary

• Scope: post and search listings by city, category and words; 30-day listings with photos; email by relay.
• Numbers: about 90 million live listings, 180 GB with their index, 360 TB of photos.
• Scale: shards by city, so a search asks one shard; New York alone, small cities together.
• Expiry: index entries carry the expiry and searches skip it; a daily job removes the entry, the photos, then the row.
• Email: a relay address per listing, both ways.
• Abuse: a rate limit per account, phone verification, flags and review.