Dropbox / Google Drive

What the interviewer is asking

"Design Dropbox" (or Google Drive) asks for a folder that stays the same on every device a person owns. Storing files is the easy part. The hard parts are moving as few bytes as possible, telling the other devices within seconds, and never losing an edit when two devices change the same file.

The walkthrough follows the 8 steps of the interview framework: scope, data, API, the busy path, numbers, services, scale and deep dives.

1. Scope: what version 1 must do

• Sync: add, change or delete a file on one device, and the others follow within seconds.
• Big files: up to 10 GB, like a long video.
• Sharing: share a folder with other people, to view or to edit.
• A few recent versions of each file, so a bad save can be undone.

Park the rest out loud: history going back months, previews and thumbnails, search inside documents, and two people typing in one document at once. That last one is Google Docs, a different design; here a file is saved whole, then synced.

2. Data: rows for facts, object storage for bytes

The app on each device, the sync agent, cuts every file into chunks of 4 MB and names each chunk by its hash, a fingerprint computed from its bytes. A 100 MB file is 25 chunks; a 10 GB video is 2,500. As in the Files and Object Storage lesson, bytes and facts live apart:

record   where            fields
file     metadata DB      path, owner, size, version, chunk list (hashes, in order)
chunk    object storage   hash → 4 MB of bytes
change   metadata DB      user, number, path, new version

The third record is the change log: every commit adds a numbered entry, and each device remembers the last number it has, its cursor. A phone that was off all weekend while the laptop changed 30 files asks for everything after its cursor and gets exactly those 30, deletes included. Comparing full file lists would fetch 100,000 rows to find 30 changes, and "changed since my clock's time" fails because clocks drift.

3. The API: upload, commit, catch up

POST /chunks/missing   {"hashes": ["…25 hashes…"]}
  → 200 OK   {"missing": ["…"], "urls": ["…signed upload URLs…"]}
PUT  <signed URL>       4 MB of bytes, straight to chunk storage
POST /files/commit     {"path": "/sales.csv", "base": 4, "chunks": ["…25 hashes…"]}
  → 200 OK   {"version": 5}
  → 409 Conflict   {"current": 5}   (someone else committed first)
GET  /changes?cursor=41
  → 200 OK   {"entries": [{"n": 42, "path": "/sales.csv", "version": 5}], "cursor": 42}

The commit carries the path, the full list of chunk hashes in order, and the base version the edit started from. Until it succeeds the new chunks are invisible, so no device ever sees half a file. Each device also keeps one connection open to a notification service (section 6). Dropbox's public API works on the same idea: a cursor, a call that lists the changes after it (files/list_folder/continue), and a long poll that waits for the next change (files/list_folder/longpoll).

4. The busy path: syncing a small edit

Ana fixes a typo in sales.csv (100 MB) on her laptop:

1. The laptop's agent sees the change, cuts the file into chunks and hashes them.
2. It asks which chunks are missing and gets signed upload URLs.
3. It uploads the missing chunks straight to chunk storage.
4. It commits the chunk list with base 4; the service saves version 5 and adds entry 42 to Ana's change log.
5. Her phone asks for the changes after its cursor and downloads the chunks it lacks.

Bytes travel only between devices and object storage, and only new chunks travel at all.

5. The numbers

From the interviewer: 500 million users with 10 GB of space each, of which the average user fills about 2 GB, and 100 million people a day saving about 10 changes each. Stored data is kept in 3 copies. A day has 86,400 seconds; round it to 100,000.

what         how                                              about
files        500 million × 2 GB = 1 billion GB                1,000 PB
on disk      1,000 PB × 3 copies                              3,000 PB
commits      100 million × 10 = 1 billion a day ÷ 100,000 s   10,000 a second (11,600 with 86,400)
busy hours   5 × 10,000                                       50,000 a second
new chunks   10,000 commits × 4 MB                            40 GB a second, 4 PB a day
metadata     500 million × 1,000 files × 1 KB a row           500 TB

Two common slips: sizing for the space handed out (15,000 PB) and forgetting the copies (1,000 PB). Passing 40 GB a second through your own servers would take about 40 of them just for the network, at about 1 GB a second each, so devices talk to storage directly. The metadata is tiny next to the files, but too big and busy for one database server.

6. Services

• Sync agent (on each device): watches the folder, cuts and hashes files, keeps the cursor.
• Metadata service: signs URLs, runs commits, serves the change log, checks access.
• Chunk storage: object storage keyed by hash.
• Notification service: holds a connection from every online device and says "changed".

Polling GET /changes every 5 seconds from 100 million online devices would be 20 million requests a second, nearly all answered "nothing new". So each device holds a long poll or a WebSocket (see Real-Time Updates): at about 100,000 connections a server, about 1,000 servers, kept apart from the metadata service. After a commit, the metadata service puts "folder changed" on a queue, so a commit never waits for the pushes.

7. Scale: splitting the metadata

At busy hours the metadata service commits about 50,000 changes a second. Say one database server handles about 5,000: the metadata must be split across at least 10 servers.

Split it by user. A commit checks the base version, saves the new version and appends to the change log in one transaction, which is why a SQL database fits (the SQL or NoSQL lesson). With a user's rows together, that transaction and every GET /changes stay on one server. Hashing file paths instead would scatter one user's files across all servers, and read replicas don't help, since each one repeats every write.

8. Deep dive: chunks and content hashes

Here is a toy file whose chunks are 4 characters instead of 4 MB. Ana changes "Rome" to "Oslo":

chunk   before   after   SHA-256, first 6 characters
1       Trip     Trip    bd13d5
2       ␣to␣     ␣to␣    877216
3       Rome     Oslo    d0d220 → 43152c   (new: the only upload)
4       ␣in␣     ␣in␣    2eb9d0
5       May.     May.    71dc54

Four hashes stay the same, so the agent uploads one chunk and commits all five hashes. On the real file, a typo fixed in one row changes one of 25 chunks: 4 MB goes up instead of 100, 25× less. A caveat: a typo fix keeps the length, but text inserted in the middle shifts every later byte, so with fixed 4 MB chunks every later chunk would change. Some systems cut chunks where the content says (a rolling hash), so only the chunks around an insert change.

Hash names give deduplication for free. When Ben saves the very same file in his own account, all 5 hashes are already stored and his laptop uploads nothing. Storage counts the files using each chunk and deletes it only when none do. If 1 chunk in 5 across all users were a copy, that would save about 600 of the 3,000 PB. Equal hashes mean equal bytes: nobody has ever found two different inputs with the same SHA-256 hash. Dropbox's API documents a content hash built the same way, from the SHA-256 of each 4 MB block.

One trade-off: across accounts, deduplication can reveal that someone already stores a file, so some services deduplicate only within one account.

9. Deep dive: syncing other devices

Ana has a laptop, a phone and a tablet that has been off since entry 39. Her laptop commits sales.csv version 5:

moment                            log        laptop   phone   tablet
start                             39 to 41   41       41      39 (off)
laptop commits sales.csv v5       adds 42    42       41      39 (off)
phone told "changed", asks 41     gets 42    42       41      39 (off)
phone downloads 1 chunk of 25     -          42       42      39 (off)
tablet back, asks 39              gets 3     42       42      42

The notification only says that something changed; the cursor says what. The tablet gets entries 40, 41 and 42, so nothing it missed is skipped. If the log was trimmed past a cursor, the device lists the whole folder once.

10. Deep dive: conflicts

On a train with no signal, Ana rewrites the intro of report.docx (version 5) on her laptop and closes it. That evening she fixes a date on her phone, online:

commit                    base   current   result
phone: date fixed         5      5         saved as version 6
laptop: new intro         5      6         conflict: 5 ≠ 6

The check is one comparison inside the commit's transaction. On a mismatch, last-write-wins would save the laptop's edit as version 7 and silently drop the phone's fix, and an automatic merge can break a .docx, a photo or a spreadsheet. The safe answer is to keep both: version 6 stays report.docx, the service answers 409, and the laptop's agent commits its edit as report (Ana's conflicted copy).docx for Ana to merge. It's the same choice as keeping both carts in the Key-Value Store lesson: offline devices keep working, and clashes are settled later. Dropbox, for one, makes such conflicted copies.

11. Deep dive: sharing

Ana shares /trip with Ben, who can edit, and Cy, who can only view. The folder keeps an access list, checked before every commit, change list and signed URL, so a commit from Cy is refused. When Ben saves beach.jpg from his laptop, every member's devices must hear: Ben's phone, Ana's laptop and phone, and Cy's phone, but not the laptop that made the change, and not Dee, who isn't on the list. A shared folder keeps its own change log, followed by each member's devices with a cursor, so its commits stay on one server too.

What breaks first

• A folder shared with thousands: one change wakes thousands of devices. The queue spreads the pushes, and devices wait a random moment before fetching.
• A notification server crashes: 100,000 devices reconnect and ask GET /changes with their cursors, so nothing is missed.
• Chunks nobody lists: deleted files and expired versions leave chunks behind; a background job deletes those whose count reaches zero.

Summary: the order to follow

1. Scope: one folder on every device, files up to 10 GB, sharing.
2. Data: file rows with a chunk list; 4 MB chunks in object storage under their hash; a change log.
3. API: signed chunk uploads, POST /files/commit with a base version, GET /changes?cursor=….
4. Busy path: a small edit to a big file; only new chunks travel.
5. Numbers: about 3,000 PB on disk, 10,000 commits a second.
6. Services: sync agent, metadata service, chunk storage, notification service.
7. Scale: the metadata split by user.
8. Deep dives: content hashes and deduplication, cursors, conflicted copies, an access list per shared folder.

The same pieces fit any app that syncs across devices: bytes in object storage, facts in rows, a log with a cursor, and a version check on every write.