Capstone: Look Before You Model in Python

This lesson starts a five-lesson project: predict which shop customers will buy again. At work you don't start with a model. You first look at the data: how many customers came back, which cells are blank, which text is messy. The starting code reads customers.csv with pd.read_csv and prints three first looks: the share who bought again, the blank cells in city and age, and the customers per city.

Cleaning: .str.strip() gives back each text without the spaces around it, and .str.title() gives it back with a capital first letter and small letters after. fillna(value) gives back the column with every blank cell (NaN) filled with value, and .median() gives back the middle value of a number column. Store the result back in the column, or the change is lost. For example, on the orders file:

orders["product"] = orders["product"].str.strip().str.lower()
orders["quantity"] = orders["quantity"].fillna(1)

Shares by group: the mean of a 0/1 column is the share of 1s. groupby(column) splits the rows into one group per value, so groupby(column)["bought_again"].mean() gives back the share who came back in each group. .round(3).to_dict() turns it into a short dictionary you can print on one line. For example, the average unit price per category:

prices = orders.groupby("category")["unit_price"].mean()
print(prices.round(2).to_dict())

When you're done, the program prints this (the last two lines come from the check, which runs after your code):

Bought again overall: 0.597
Blank cells: {'city': 9, 'age': 14}
Cities: {'Cluj': 50, 'Lyon': 48, 'Rome': 47, 'Porto': 46, 'Madrid': 45, 'Berlin': 43, 'madrid': 6, 'Rome ': 3, ' cluj ': 2, 'BERLIN': 1}
newsletter {'no': 0.485, 'yes': 0.737}
returned_item {0: 0.611, 1: 0.486}
city {'Berlin': 0.523, 'Cluj': 0.519, 'Lyon': 0.688, 'Madrid': 0.627, 'Porto': 0.609, 'Rome': 0.62, 'Unknown': 0.556}
Stands out: newsletter readers come back far more often than the rest.
Blank cells now: {'city': 0, 'age': 0}
Average age: 37.38

Your task: below the starting prints, clean the city text, fill the blank cities with "Unknown" and the blank ages with the median age. Then loop over "newsletter", "returned_item" and "city", print the share who bought again in each group, and print one sentence on what stands out.

Assignment
Follow the Coding Tutorial to clean customers and print the share who bought again by newsletter, by returned item and by city.

Hint
Clean before you group: with the messy text, "madrid" and "Madrid" would be two groups. The two lines inside the loop are indented under the for.