Shopify

The problem

"Design Shopify." Anyone can open an online shop: Luna sells candles at luna-candles.com, Bo sells ceramics at bo-ceramics.com, and the platform runs every storefront, the checkout and the payments. Apps by other companies plug in.

It reuses the Payment System's pattern: one charge per idempotency key, a double-entry ledger, payouts to sellers. What's new is many small shops on shared machines: each shop's data kept apart, a storefront on each shop's own domain, a sale in one shop that must not slow the others, and outside apps. Park card readers and prices in many currencies.

The numbers

From the interviewer: 2 million shops, about 10 million orders a day in all, and about 100 page views per order. A day is about 100,000 seconds (86,400 really); busy hours run about 5× the average.

• Orders per shop: 10 million ÷ 2 million = 5 a day on average. Most shops are tiny, so they share servers and databases: a multi-tenant platform.
• Pages: 10 million × 100 = 1 billion a day, 10,000 a second on average (about 11,600 with 86,400), so about 50,000 a second at busy hours.
• The spread: say the biggest shop gets 50,000 orders a day, 10,000 times the average.

Many shops: a shop id on every row

table       columns
products    shop_id, id, price, …
orders      shop_id, id, total, …
customers   shop_id, id, email, …

Split the data by shop: every row carries its shop id, all of a shop's rows live on one shard, and every query filters by the shop id. A shop's pages and admin only ask about that shop, so one shard answers them. Add the filter in one shared data layer, not in each query, so it's never forgotten.

A directory, shop → shard, says where each shop lives. A shop that grows huge, like the one with 50,000 orders a day, moves to a shard of its own: copy its rows while it keeps selling, then change its entry. Hashing each row's own id would scatter Luna's orders over every shard (scatter-gather); a database per shop means 2 million databases, nearly all idle at 5 orders a day.

Storefronts on their own domains

Luna's and Bo's domains both point at the platform. A server reads the request's host name and looks up the shop: say 2 million domains of about 100 bytes, 200 MB, a table every server keeps in memory. Each domain also needs an HTTPS certificate, issued by the platform.

A storefront page is the same for every visitor of that shop, so a CDN caches it, under host + path:

key                 page
luna-candles.com/   Luna's home page
bo-ceramics.com/    Bo's home page

Keyed by path alone, both home pages are /, and Bo's visitors would get Luna's page. Say the CDN answers 9 pages in 10: of 50,000 a second at busy hours, about 5,000 reach the servers. Carts and checkout skip the cache, and editing a product clears only that shop's pages.

One shop's flash sale

At noon Luna's shop goes viral: 10,000 cart and checkout requests a second hit the shard she shares with thousands of shops, which together send about 1,000. Say a shard answers about 5,000 a second. Give each shop its own limit, a token bucket keyed by the shop id as in the Rate Limiter; say 1,000 requests a second:

one second at noon   arrive   get in
Luna                 10,000   1,000
the other shops      1,000    1,000
the shard            11,000   2,000 of 5,000

Luna's other requests are told "try again in a moment". One shared limit fails: first come, first served, 11,000 requests compete for 5,000 places, and more than half of the other shops' requests are turned away. Moving her to her own shard takes hours: too slow for a sale under way. Her stock is one conditional update, as in the Flash Sale, so she never oversells.

One checkout for every shop

Shops own what buyers see: the theme, prices, photos, shipping. The platform owns everything that touches card data or money. A theme is page code anyone may write, so card details must never pass through it: one checkout for every shop keeps card data in one place, under PCI DSS. Behind it is the Payment System: an idempotency key per checkout, a ledger with an account for each shop, and a nightly payout of each balance, keyed by shop and day.

Apps and webhooks

An email app installed in 100,000 shops wants every new order. Asking each shop every 10 seconds would be 10,000 requests a second, each shop asked about 10,000 times a day for its 5 orders. Instead the platform sends a webhook for each new order, as in the Webhooks Service: signed, through a queue per app, retried with backoff. The app then calls the shop's API with a token for that one shop and app, with scopes like "read orders", limited per app per shop. Apps never read the database: the API stays the same when tables change.

Same pattern as the Payment System

All the money handling carries over: save the payment before calling the provider, retry with the same key after a timeout, a nightly reconciliation, and a ledger fixed with new entries, never edits. Each shop is one more account.

Summary

• Scope: millions of shops, storefronts on their own domains, one checkout, apps.
• Data: a shop id on every row; shards by shop, huge shops on their own.
• API: a shop API with a token per app and shop; webhooks to apps.
• Busy path: a storefront page, host → shop, cached at the CDN under host + path.
• Numbers: 2 million shops, about 5 orders a day each; 50,000 pages a second at busy hours.
• Services: one checkout for every shop, a ledger account each.
• Scale: a limit per shop, so one sale can't slow the rest.