Late Rows in Python

Some rows arrive late. Order 5036 was placed on 2026-05-02 at 21:47, but it came in the export of 05-04, orders/2026-05-04.csv. If each file's rows go into the folder of the file's day, 05-02 is one order short and 05-04 has one too many, for good. Late rows are normal at work (an export that missed an order, a phone that was offline), so a data engineer files each row under the day it happened, the day in ordered_at, not the day it arrived.

Each row's day: .str[:10] cuts every text in a column to its first 10 characters, so "2026-05-02 21:47:00" becomes "2026-05-02". .unique() lists each value of a column once. A comparison such as days == "2026-05-02" is True for that day's rows and filters df like any pandas filter.

Merge with what is there: 05-04's run now also writes into the 05-02 folder, which already holds 12 orders; writing it from scratch would leave only order 5036. So when the file exists (os.path.exists(path)), read it with pd.read_parquet(path) and put the new rows after its rows with pd.concat, which takes a list of DataFrames and stacks their rows. Then drop_duplicates("order_id", keep="last") keeps one row per order: running a day again adds nothing twice, and when an order comes twice, the newest copy wins.

times = pd.Series(["2026-06-10 09:15:00", "2026-06-09 23:58:00"])
times.str[:10]   # "2026-06-10", "2026-06-09"

old = pd.DataFrame({"id": [1, 2], "qty": [1, 1]})
new = pd.DataFrame({"id": [2, 3], "qty": [4, 1]})
both = pd.concat([old, new]).drop_duplicates("id", keep="last")
# ids 1, 2, 3: id 2 keeps qty 4, the newer copy

Your code runs 05-01 to 05-04. The check prints the orders in each day's folder and where order 5036 sits, then calls run("2026-05-04") again: nothing may change.

date | orders | first_order | last_order
2026-05-01 | 10 | 5001 | 5010
2026-05-02 | 13 | 5011 | 5036
2026-05-03 | 10 | 5023 | 5033
2026-05-04 | 8 | 5034 | 5042
(4 rows)
date | order_id | ordered_at
2026-05-02 | 5036 | 2026-05-02 21:47:00
(1 row)
The check calls run("2026-05-04") again:
date | orders | first_order | last_order
2026-05-01 | 10 | 5001 | 5010
2026-05-02 | 13 | 5011 | 5036
2026-05-03 | 10 | 5023 | 5033
2026-05-04 | 8 | 5034 | 5042
(4 rows)

Your task: extract, transform and load(rows, day) are written for you; load writes rows into warehouse/orders/date=<day>/orders.parquet, and your code already runs 05-01 to 05-04. In run, find each row's day and call load once per day with that day's rows. In load, when the day's file is already there, put its rows before the new ones, then drop repeated order ids, keeping the last copy.

Assignment
Follow the Coding Tutorial to load each order into the folder of the day it was placed, merged with the orders already there.

Hint
Without the merge, 05-02 ends with 1 order, only 5036. With the merge but without drop_duplicates, the check's second run grows 05-02 to 14 rows and 05-04 to 16.