Logistic Regression from Scratch in Python

An interviewer asks: "Implement logistic regression with numpy only, then predict 0 or 1 for each row." The idea in one sentence: take the linear score X @ w + b, squeeze it into a chance between 0 and 1 with the sigmoid, and train w and b with gradient descent, the same loop as linear regression.

The sigmoid of a score z is 1 / (1 + e^−z): a score of 0 gives 0.5, big scores give almost 1 and very negative ones almost 0. np.exp(z) gives e to the power z, for every number of an array at once. Comparing an array with a number gives one True or False per value, and .astype(int) turns them into 1 and 0:

ages = np.array([15, 22, 18, 40])
adult = (ages >= 18).astype(int)
print(adult)          # [0 1 1 1]
print(np.exp(0.0))    # 1.0

Training lowers the log loss, and its gradients are short. With the chances p and the labels y, the gradient for the weights is X.T @ (p - y) divided by the number of rows, and the gradient for b is the mean of p - y. Each turn of the loop subtracts lr times each gradient. To predict, a row is a 1 when its chance is over 0.5.

With lr = 0.3 and 20000 turns, your program must print these lines (the last three come from the lesson's check, which compares your model with scikit-learn's LogisticRegression):

w: [0.932 0.914] b: -5.707
Chances: [0.021 0.051 0.253 0.116 0.462 0.678 0.845 0.674 0.93  0.972]
Predictions: [0 0 0 0 0 1 1 1 1 1]
Real labels: [0 0 1 0 0 0 1 1 1 1]
sigmoid of -10, 0 and 10: [0.  0.5 1. ]
scikit-learn's LogisticRegression on the same rows: w: [0.932 0.914] b: -5.707
Same predictions as scikit-learn on 60 random rows: 60 of 60

Your task: the 10 rows, their labels and most of fit_logistic are in the editor: w starts as np.zeros(X.shape[1]), one zero per column, and the two updates are written. Add sigmoid, then the chances p and the two gradients inside the loop, then turn chances into predictions with a 0.5 cut.

Assignment
Follow the Coding Tutorial to write sigmoid, the gradients inside fit_logistic and the 0.5 cut.

Hint
Mind the minus in np.exp(-z): without it, big scores would give chances near 0. Put p - y in parentheses, both after X.T @ and before .mean().