Start learning free

LESSON 1 OF 107 · DATA ENGINEERING: FILES AND FORMATS

CSV's Traps in Python

Practice

A CSV file is plain text: values with commas between them, and nothing that says which column holds numbers. When pd.read_csv reads one, pandas guesses each column's type, its dtype, from the values it finds; df.dtypes lists the guesses. One odd value is enough to make a guess go wrong, and no warning appears.

Trap 1: a word in a number column. If one row says two instead of 2, the whole column comes back as text (str), and .sum() glues the values together instead of adding them. pd.to_numeric turns a column into numbers; with errors="coerce", a value that isn't a number becomes NaN (missing) instead of stopping the program with an error:

counts = pd.Series(["3", "five", "2"])
counts = pd.to_numeric(counts, errors="coerce")   # 3.0, NaN, 2.0

Trap 2: a blank in a whole-number column. pandas reads a blank as NaN, and NaN is a float, so ids like 38 come back as 38.0 (float64). The dtype= argument of pd.read_csv takes a dict from column name to type and uses it instead of guessing. "Int64", with a capital I, is pandas' whole-number type that can hold a missing value, shown as <NA>: for example pd.read_csv("visits.csv", dtype={"user_id": "Int64"}).

Why a data engineer needs this: a pipeline that trusts the guesses adds up text, or sends ids like 38.0 into every report and export after it, and nothing stops it. You set the types where the data comes in, so every later step can count on them.

Your program reads two of the shop's daily exports: in orders/2026-05-03.csv order 5030's quantity is two, and in orders/2026-05-06.csv order 5058 has a blank customer_id. It prints the dtypes pandas guessed for 05-03, and the lesson's check prints the rest:

order_id       int64
customer_id    int64
product          str
quantity         str
unit_price     int64
status           str
ordered_at       str
dtype: object
quantity now: float64
items sold on 05-03: 20.0
orders with no quantity: [5030]
customer_id without dtype: float64
customer_id with your dtype: Int64
customers on 05-06: [38, 65, 272, <NA>, 84, 157, 95, 189, 209, 9]

Your task: read orders/2026-05-03.csv into day3 and print its dtypes, turn day3["quantity"] into numbers with pd.to_numeric and errors="coerce", then read orders/2026-05-06.csv into day6 with customer_id set to "Int64".


Assignment
Follow the Coding Tutorial to read two daily exports and take control of their column types.


Hint
pd.to_numeric gives back a new column: store it in day3["quantity"] again. dtype= takes a dict from column name to type name, and "Int64" needs its capital I.


STEP 1 OF 4
    Ctrl + Enter to check

    De Csv Traps in Python

    Keep going with Python1,600+ step-by-step lessons, from your first line of code to coding interviews. Free to start.
    Start learning free