Batches: Dataset and DataLoader in Python

Until now, each epoch made one update from all the rows at once. With lots of data that is slow, and one update per pass is few. The usual way is to cut the rows into batches, small slices, and make one update per batch.

TensorDataset(x, y) pairs two tensors row by row: dataset[i] gives back the pair (x of row i, y of row i). DataLoader(dataset, batch_size=..., shuffle=True, generator=...) gives back something you loop over: each turn hands out one batch of x rows with their y rows. With shuffle=True the rows come in a new order every epoch, and a seeded torch.Generator() makes that order the same on every run. A small example on other tensors, without shuffling:

a = torch.tensor([[1.0], [2.0], [3.0], [4.0], [5.0]])
pairs = TensorDataset(a, a * 10)
print(pairs[1])                  # (tensor([2.]), tensor([20.]))
for ab, bb in DataLoader(pairs, batch_size=2):
    print(ab.tolist())           # [[1.0], [2.0]], then [[3.0], [4.0]], then [[5.0]]

Your 300 rows in batches of 64 make 5 batches: 64, 64, 64, 64 and 44. So each epoch now makes 5 updates instead of 1. The print shows the loss of the last batch of each epoch, so the number wobbles a little. Your program should print:

epoch 0 loss: 1.529
epoch 1 loss: 0.209
epoch 2 loss: 0.025
epoch 3 loss: 0.011
epoch 4 loss: 0.012
epoch 5 loss: 0.009
epoch 6 loss: 0.007
epoch 7 loss: 0.015
epoch 8 loss: 0.016
epoch 9 loss: 0.01
w: 3.008 b: 2.007

The check then counts your batches and compares with the old way: loss 0.012 on all 300 rows, against 0.093 for one update per epoch.

Your task: below the optimizer, build dataset and loader (batches of 64, shuffled, generator seeded with 0). Inside the epoch loop, above the print, loop over the batches with for xb, yb in loader: and run the five training lines on xb and yb.

Assignment
Follow the Coding Tutorial to build the dataset and the loader, then train on one batch at a time.

Hint
The five lines go eight spaces in, inside the batch loop; the print stays four spaces in, so it runs once per epoch. Use xb and yb in the five lines, not x and y.