Capstone: Say What You Found in Python

A model only helps once you can tell someone what it found. This lesson ends the project with a short printed report: how good the model is against guessing, which columns it leans on, and the 5 customers most likely to come back.

Against guessing. y_test holds 1 for each test customer who came back and 0 for the rest, so y_test.mean() gives back the share of 1s: exactly how often "always guess yes" would be right. The test accuracy, 0.813, only means something next to that number.

What matters. permutation_importance(model, X, y, n_repeats=10, random_state=0) shuffles one column at a time, 10 times each, and measures how much the accuracy drops: the columns the model needs most drop it most. It gives back a result whose importances_mean holds one average drop per column, in the order of the columns. pd.Series(values, index=names) puts names on numbers, .sort_values(ascending=False) puts the biggest first, .head(n) keeps the first n and .to_dict() turns them into a dictionary:

drops = pd.Series([0.2, 0.5, 0.1], index=["a", "b", "c"])
print(drops.sort_values(ascending=False).head(2).to_dict())   # {'b': 0.5, 'a': 0.2}

Who comes back. Column 1 of model.predict_proba(X) is each row's chance of a 1. df.nlargest(5, "chance") gives back the 5 rows with the biggest "chance", biggest first (on a product table, products.nlargest(2, "price") gives back the two most expensive products), and .to_string(index=False) prints a table without the row labels on the left.

Your script must print:

Test accuracy: 0.813
Always guessing yes: 0.6
Matters most: {'visits': 0.097, 'days_since_last': 0.053, 'spent': 0.049}
 customer_id  visits  days_since_last  spent  chance
         139      22                3    435   1.000
         261      18               16    390   0.999
          53      17               54    505   0.999
         209      18                8    357   0.998
         177      16               35    502   0.998

Your task: below the test accuracy, print the always-yes share of y_test, measure the permutation importance of model on the test rows, and print the 3 columns that matter most as a dictionary. Then add a "chance" column to customers for all 300 customers, keep the 5 biggest in top, and print their customer_id, visits, days_since_last, spent and chance, rounded to 3 digits.

Assignment
Follow the Coding Tutorial to print the report: the model against guessing, the columns that matter most, and the 5 customers most likely to come back.

Hint
importances_mean has one number per column of X, in the same order, so X.columns fits as the index. X holds all 300 customers in the same order as customers, so its chances fit as a new column.