Write, Then Rename in Python

Writing a file is not instant. to_parquet(path) opens the file, empties it, then fills it bit by bit. If the program dies halfway (the disk is full, the machine restarts), the file is left half written: the old data is gone and the new data is cut off. A dashboard or the next pipeline that reads it then gets an error, and the day stays broken until someone reloads it.

The fix is an atomic write: write the new file under a temporary name next to the real one, then swap it in. os.replace(src, dst) renames src to dst, replacing the old dst, in one step: a reader sees the old file or the new one, never half of one. The same idea with a small text file:

with open("report.txt.tmp", "w") as f:
    f.write("orders: 12")
os.replace("report.txt.tmp", "report.txt")

Keep the temporary file in the same folder: os.replace is one step only on the same disk. A crash can leave the .tmp file behind, but readers ask for orders.parquet or *.parquet, so they never open it, and the next run writes over it. Table formats such as Delta Lake, later in the course, build on the same idea for whole tables.

The hidden code saves an older load of 05-02 (its 10 paid orders) as warehouse/orders/date=2026-05-02/orders.parquet, then makes your first to_parquet stop halfway with "No space left on device". The old file must still read whole after the crash; when the check runs 05-02 again with no crash, the file must hold all 12 orders:

The load stopped: [Errno 28] No space left on device
['orders.parquet', 'orders.parquet.tmp']
status | orders
paid | 10
(1 row)
The check runs run("2026-05-02") again, with no crash:
['orders.parquet']
status | orders
cancelled | 2
paid | 10
(2 rows)

Your task: in load, set tmp to final + ".tmp", write df to tmp with to_parquet (no index), then put it in place of final with os.replace(tmp, final).

Assignment
Follow the Coding Tutorial to make load write to a temporary file and swap it in.

Hint
Only the last line of load touches final: the write goes to tmp, and os.replace comes after the write has finished.