Lesson 32 · Python For Time Series
Walk-Forward Validation in Python: A Clear Guide for Time Series Model Evaluation
Welcome! Today, we will explore how to check time series model performance using walk-forward validation. This is a must-have skill in data science and…
- CoursePython For Time Series
- Lesson32 of 30
- Video15 min
- FormatJupyter notebook · 15 code cells
What you'll learn
Data
No separate download needed — the notebook creates or downloads everything it uses.
📓 Full notebook
Download .ipynb
Walk-Forward Validation in Python: A Beginner-Friendly Guide#
Welcome!
Today, we will explore how to check time series model performance using walk-forward validation.
This is a must-have skill in data science and forecasting.
We will use Python, relatable datasets, and clear step-by-step examples.
Ready? Let us get started!
import warnings
warnings.filterwarnings("ignore")
# Suppress warnings for a smooth learning experience
What is Walk-Forward Validation?#
Walk-forward validation is a way to check how well your model predicts on new, unseen data.
It tests predictions just one step ahead at a time, always moving forward.
This is important for real-world forecasting, because we do not get to peek at the future when making predictions!
# Data setup: Let us get a real-world time series!
import pandas as pd
url = "https://raw.githubusercontent.com/jbrownlee/Datasets/master/airline-passengers.csv"
data = pd.read_csv(url, parse_dates=['Month'])
print("Shape:", data.shape)
print("First 5 rows:")
print(data.head())
# Plot the time series to see the pattern
import matplotlib.pyplot as plt
plt.figure(figsize=(10, 4))
plt.plot(data['Month'], data['Passengers'], marker='o')
plt.title("Monthly Airline Passengers")
plt.xlabel("Month")
plt.ylabel("Passengers")
plt.grid(True)
plt.tight_layout()
plt.show()
How Walk-Forward Validation Works#
- Train with data at the start.
- Predict the next point.
- Compare prediction to actual value.
- Move one step forward and repeat.
Imagine walking along a timeline, always using what you have seen so far to predict the next step!
# Let us split the data into training and testing sets
from sklearn.model_selection import train_test_split
values = data['Passengers'].values
train_size = int(len(values) * 0.7)
train, test = values[:train_size], values[train_size:]
print(f"Training points: {len(train)} | Testing points: {len(test)}")
# Create a simple prediction: Use the last seen value (naive forecast)
predictions = []
history = list(train)
for t in range(len(test)):
yhat = history[-1]
predictions.append(yhat)
history.append(test[t])
print(f"First 5 predictions: {predictions[:5]}")
# Let us see how accurate our predictions are
from sklearn.metrics import mean_squared_error
import numpy as np
mse = mean_squared_error(test, predictions)
rmse = np.sqrt(mse)
print(f"RMSE: {rmse:.2f}")
# Visualize: Compare predictions to real values
plt.figure(figsize=(10,4))
plt.plot(data['Month'][-len(test):], test, label="Actual")
plt.plot(data['Month'][-len(test):], predictions, label="Predicted", linestyle="--")
plt.title("Walk-Forward Predictions vs. Actuals")
plt.xlabel("Month")
plt.ylabel("Passengers")
plt.legend()
plt.grid(True)
plt.tight_layout()
plt.show()
Trying a Better Model#
Our naive rule is a good baseline, but can we do even better?
Let us upgrade to a moving average, which smooths out sudden jumps.
# Walk-forward with a simple moving average (window=3)
window = 3
pred_ma = []
history_ma = list(train)
for t in range(len(test)):
if len(history_ma) < window:
yhat_ma = history_ma[-1]
else:
yhat_ma = np.mean(history_ma[-window:])
pred_ma.append(yhat_ma)
history_ma.append(test[t])
print("First 5 moving average predictions:", pred_ma[:5])
# Check the accuracy of the moving average forecast
mse_ma = mean_squared_error(test, pred_ma)
rmse_ma = np.sqrt(mse_ma)
print(f"RMSE (Moving Average): {rmse_ma:.2f}")
# Plot moving average predictions vs. real values
plt.figure(figsize=(10,4))
plt.plot(data['Month'][-len(test):], test, label="Actual")
plt.plot(data['Month'][-len(test):], pred_ma, label="Moving Average Prediction", linestyle="--")
plt.title("Moving Average Walk-Forward Predictions")
plt.xlabel("Month")
plt.ylabel("Passengers")
plt.legend()
plt.grid(True)
plt.tight_layout()
plt.show()
# Using a smart model: ARIMA (quick version for demonstration)
from statsmodels.tsa.arima.model import ARIMA
history_arima = list(train)
pred_arima = []
for t in range(len(test)):
model = ARIMA(history_arima, order=(1,1,0))
model_fit = model.fit()
yhat_arima = model_fit.forecast()[0]
pred_arima.append(yhat_arima)
history_arima.append(test[t])
print("First 5 ARIMA predictions:", pred_arima[:5])
# Check ARIMA prediction error
mse_arima = mean_squared_error(test, pred_arima)
rmse_arima = np.sqrt(mse_arima)
print(f"RMSE (ARIMA): {rmse_arima:.2f}")
Walk-Forward Validation: Benefits and Best Practices#
- It mimics how we use models in real life, predicting one step ahead.
- Great at showing when a model starts making big mistakes.
- Easy to compare models fairly, since no future data is leaked!
Always remember to update your "history" after each step, so your predictions stay realistic.
# Try it yourself! Walk-forward on a different dataset
url2 = "https://raw.githubusercontent.com/jbrownlee/Datasets/master/shampoo.csv"
data2 = pd.read_csv(url2)
print("Shampoo sales preview:\n", data2.head())
values2 = data2['Sales'].values
size2 = int(len(values2) * 0.7)
train2, test2 = values2[:size2], values2[size2:]
preds2 = []
hist2 = list(train2)
for t in range(len(test2)):
yhat2 = np.mean(hist2[-3:]) if len(hist2) >= 3 else hist2[-1]
preds2.append(yhat2)
hist2.append(test2[t])
rmse2 = np.sqrt(mean_squared_error(test2, preds2))
print(f"RMSE: {rmse2:.2f}")
# Challenge: Let the user pick a forecast method
print("Choose method: 1 for last value, 2 for moving average")
choice = input("Enter 1 or 2: ")
if choice == "1":
print("You picked: Last value (naive)")
preds_challenge = []
hist_challenge = list(train)
for t in range(len(test)):
yhat = hist_challenge[-1]
preds_challenge.append(yhat)
hist_challenge.append(test[t])
elif choice == "2":
print("You picked: Moving average (window=3)")
preds_challenge = []
hist_challenge = list(train)
for t in range(len(test)):
if len(hist_challenge) < 3:
yhat = hist_challenge[-1]
else:
yhat = np.mean(hist_challenge[-3:])
preds_challenge.append(yhat)
hist_challenge.append(test[t])
else:
print("Please enter only 1 or 2.")
if choice in ["1", "2"]:
error = np.sqrt(mean_squared_error(test, preds_challenge))
print(f"Your selected method RMSE: {error:.2f}")
Recap#
- You learned the basics of walk-forward validation for time series.
- You saw how to use naive, moving average, and ARIMA models.
- You measured accuracy and compared results.
You now have the essential building blocks to evaluate your own forecasts!
Next Steps & Call to Action#
Try building your own challenge with a new dataset.
Change forecast rules and experiment to see what works best.
If you found this lesson helpful, subscribe for more friendly guides on YouTube!
Happy forecasting!
Found this useful?
All lessons, notebooks and datasets here are free. If they helped you, a coffee keeps new lessons coming.



