Start learning free

LESSON 111 OF 165 · MACHINE LEARNING: TIME SERIES

Split by Time, Never at Random in Python

Practice

A forecast is made today, about weeks that haven't happened yet. To test one honestly, every test week must come after every training week. train_test_split doesn't know that: it picks test rows at random, so the model trains on weeks before and after each test week. It has seen the future, and its error looks better than it will ever be.

The honest cut is by position. .iloc[:-8] gives back every row but the last 8, in order, and .iloc[-8:] gives back the last 8. Nothing is shuffled. On a small series:

days = pd.Series([5, 7, 6, 8, 9, 11])
early, late = days.iloc[:-2], days.iloc[-2:]
print(early.tolist(), late.tolist())   # [5, 7, 6, 8] [9, 11]

The starting code builds the weekly table with lag_1 and lag_4, splits it at random, fits a KNeighborsRegressor and prints two lines: the months of the test weeks and the mean absolute error (the average miss, in revenue). X_test.index holds the test weeks' dates; .month gives back each date's month number and .tolist() turns them into a plain list. Calling fit again starts the model over. When you're done, the script prints:

Random split, test months: [8, 10, 8, 3, 4, 7, 8, 4]
Random split MAE: 374.8
Time split, test months: [10, 10, 10, 11, 11, 11, 11, 11]
Time split MAE: 913.2

The time split's error is more than twice as big. It's the number to trust: it's what the model really does on weeks it hasn't seen.

Your task: below the random split, cut X and y by time (the last 8 weeks are the test), fit model again on the training rows, and print the test months and the mean absolute error, rounded to 1 decimal.


Assignment
Follow the Coding Tutorial to split the weeks by time, fit the model again and print its honest test error.


Hint
Cut X and y at the same iloc positions, so each week keeps its revenue. fit starts over, so the model forgets the random split's rows.


STEP 1 OF 5
    Ctrl + Enter to check

    Ts Split in Python

    Keep going with Python1,600+ step-by-step lessons, from your first line of code to coding interviews. Free to start.
    Start learning free