Gradients for Free: autograd in Python

In the from-scratch chapter you worked out the slopes of the loss by hand. For a prediction w * x + b and a squared error, the slope for w is 2 * error * x and the slope for b is 2 * error. PyTorch can do this calculus for you. It's called autograd.

When you make a tensor with requires_grad=True, PyTorch records every calculation made with it. Call .backward() on the final one-number result, and PyTorch works out the slope (the gradient) for every such tensor. It stores each slope in that tensor's .grad. .item() gives back the plain Python number inside a one-number tensor. Here it is on another formula:

a = torch.tensor(2.0, requires_grad=True)
area = a ** 2
area.backward()
print(a.grad.item())   # 4.0, the slope 2 * a

In this lesson, one data point (x = 3, with the real answer y = 11) meets a line that starts at w = 1 and b = 0. The prediction is 3, the error is 3 - 11 = -8 and the squared error is 64. Your program must print the lines below (the last three come from the lesson's check):

w.grad: -48.0 b.grad: -16.0
loss: 64.0
by hand, 2 * error * x and 2 * error: -48.0 -16.0
after a second backward(), w.grad adds up to -96.0

The last line shows something to remember: backward() adds to .grad. It doesn't replace what's already there.

Your task: below y, make w (1.0) and b (0.0) with requires_grad=True. Store the squared error (w * x + b - y) ** 2 in loss, call loss.backward() and print both gradients.

Assignment
Follow the Coding Tutorial to let PyTorch work out the slopes of a squared error and print them.

Hint
Only the tensors that training will change, w and b, need requires_grad=True. x and y are data. Read .grad after backward(), because before it runs, .grad is None.