How Wrong? The Loss in Python

Your network gives one answer per row, and y holds the right answers. Before it can learn, you need one number that says how wrong it is: the loss. Training will mean finding weights that make this number smaller.

The usual loss is the mean squared error. Take each answer minus its right answer (the error), square it, and average the squares. Squaring stops errors like 0.5 and -0.5 from cancelling out, and it makes big misses count much more: a miss by 0.2 costs 0.04, four times the 0.01 of a miss by 0.1. The loss is 0 only when every answer is exactly right.

Two numpy tools do the work. ** on an array raises every item to a power, so a ** 2 gives back each item squared. np.mean(a) gives back the average of all the numbers in a:

a = np.array([-3, 1, 2, 2])
print(a ** 2)           # [9 1 4 4]
print(np.mean(a ** 2))  # 4.5

The starting code runs the two-layer network from the last lesson with two sets of weights and prints their answers. With your loss lines added, the script prints this (the last three lines come from the check, which tries your mse on answers of its own):

y: [0 1 1 0]
first weights: [0.736 0.704 0.877 0.816]
loss: 0.3276
second weights: [0.007 0.992 0.992 0.007]
loss: 0.0001
loss of y itself: 0.0
all 0.5: 0.25
each off by 0.1: 0.01
each off by 0.2: 0.04

Your task: below forward, write a function mse(out, y) that returns the mean squared error. Print the loss after each set of weights, rounded to 4 decimals, and at the end print the loss of y compared with itself.

Assignment
Follow the Coding Tutorial to write mse and print three losses.

Hint
out - y subtracts two whole arrays item by item, so you need no loop. Round a loss with round(…, 4).