Logistic Regression in Python

Linear regression predicts a number, like spent. But bought_again is a yes or a no: 1 if the customer bought again, 0 if not. Logistic regression decides like linear regression: one weight per feature, all added up with an intercept. Then it squeezes that weighted sum into a chance between 0 and 1. A sum of 0 gives 0.5, big sums get close to 1 and very negative sums get close to 0. When the chance of yes is above 0.5, the model predicts 1.

LogisticRegression(max_iter=2000), from sklearn.linear_model, makes the model. It learns its weights in rounds, and max_iter=2000 lets it take up to 2,000 of them, so it can finish instead of stopping early with a ConvergenceWarning. model.fit(X, y) learns the weights. model.predict_proba(rows) gives back a table with one row per customer and two columns: the chance of a 0, then the chance of a 1. The two add up to 1. model.coef_ gives back the weights as one row, so model.coef_[0] is one weight per feature, in the order of your columns. A weight above 0 pushes the chance of yes up as its feature grows; a weight below 0 pushes it down.

Here are the same tools on another label, whether a shopper opens an email:

clf = LogisticRegression(max_iter=2000)
clf.fit(X_people, opened)            # opened is 1 or 0 for each shopper
table = clf.predict_proba(X_people)  # one row per shopper: [chance of 0, chance of 1]
print(table[:, 0].round(2))          # every row, column 0 only: the chances of NOT opening

Your code prints the chances of yes for the first 5 test customers, then each feature with its weight. .round(3) rounds every number in an array to 3 decimals. The check adds the test accuracy:

Chances of yes: [0.997 0.726 0.089 0.109 0.379]
visits 0.268
orders 0.278
spent 0.004
days_since_last -0.014
returned_item -0.924
Test accuracy: 0.733

visits, orders and spent push toward yes; days_since_last and returned_item push toward no. Don't compare the sizes yet: spent's 0.004 is per dollar, and the next lesson fixes that.

Your task: the data, the 5 features and the split into 225 training and 75 test customers are ready. Below the split, make a LogisticRegression(max_iter=2000) in model and fit it on the training rows. Print "Chances of yes:" with column 1 of predict_proba for X_test[:5], rounded to 3 decimals. Then keep model.coef_[0] in weights and print each feature with its weight, rounded to 3 decimals.

Assignment
Follow the Coding Tutorial to fit a logistic regression on the shop's customers, print its chances of yes for 5 test customers and print the weight it learned for each feature.

Hint
Column 1 of predict_proba holds the chances of yes: [:, 1] keeps it for every row. For the weights, loop over range(len(features)) and print features[i] next to round(weights[i], 3).