A load copies rows from a source into a target. Here the source is raw_orders, the week's files as they came, and the target is stg_orders, the table the load builds from it with real numbers. A bug in a load rarely raises an error: a stray WHERE or a wrong join just leaves rows out, and every report built on the target is quietly wrong. That is why a data engineer reconciles after each load: compute the same totals on both sides and check that they are equal.
Two totals catch most trouble: the row count, COUNT(*), finds rows that were lost or doubled, and the amount, SUM(quantity * unit_price), finds values that changed on the way. The source is all text, so cast it first: TRY_CAST(x AS INT) gives NULL when the text is not a number (like the quantity "two"), and SUM skips NULLs. A query that returns one row with two values comes back from fetchone() as a pair. An example on other numbers:
query = (
"SELECT COUNT(*), MAX(CAST(unit_price AS INT)) "
"FROM raw_orders "
)
week = con.execute(query).fetchone()
print("rows:", week[0], "top price:", week[1])
The starting code builds stg_orders with a bug: it keeps only the paid orders. It also gives you compare(rule, files, loaded), which prints rule: PASS (n) when the two numbers are equal and rule: FAIL (… in the files, … loaded) when they are not. After your lines, the hidden check fixes the load, runs your second query again and compares once more. The program prints:
row count: FAIL (78 in the files, 67 loaded) amount: FAIL (2613 in the files, 2284 loaded) With the load fixed: row count: PASS (78) amount: PASS (2613)
Your task: write a query for the files' row count and amount FROM raw_orders and keep its row in files, then the same two numbers FROM stg_orders kept in loaded, and call compare for "row count" and for "amount".
Assignment
Follow the Coding Tutorial to compare the files and the loaded table on rows and amount.
Hint
Cast only on the source side: stg_orders already holds numbers. files[0] is the row count and files[1] the amount; pass the files' number before the loaded one.
Free plan: 30 tutor messages. PRO is unlimited.
