Mathew K AnalyticsLesson 50 · Python for Retail E-commerce Analytics
Evaluating Forecast Accuracy: Techniques for Retail E-commerce Analytics in Python
In this lesson, we will learn how to evaluate forecast accuracy using real retail datasets. Accurate forecasts are essential for planning inventory,…
- CoursePython for Retail E-commerce Analytics
- Lesson50 of 43
- Video25 min
- FormatJupyter notebook · 20 code cells
What you'll learn
- Core Concepts for Retail Forecast Accuracy
- Beginner Example 1: Calculate Forecast Error
- Beginner Example 2: Calculate MAE (Mean Absolute Error)
- Beginner Example 3: Visualize Actual vs Forecasted Sales
- Intermediate Example 1: Compute MAPE (Mean Absolute Percentage Error)
- Intermediate Example 2: Calculate RMSE (Root Mean Squared Error)
- Intermediate Example 3: Rolling Window Forecasting
- Advanced Example 1: Segment Accuracy by Country
Data
No separate download needed — the notebook creates or downloads everything it uses.
📓 Full notebook
Download .ipynbEvaluating Forecast Accuracy in Retail and E-Commerce#
- In this lesson, we will learn how to evaluate forecast accuracy using real retail datasets.
- Accurate forecasts are essential for planning inventory, optimizing marketing, and maximizing sales.
- You will learn how to measure and interpret errors in sales forecasts so that you can make more informed business decisions.
- By the end, you will analyze performance, spot errors, and suggest improvements for retail sales forecasting.
import pandas as pd
import numpy as np
import warnings
warnings.filterwarnings('ignore')
Core Concepts for Retail Forecast Accuracy#
- Retail datasets typically track transactions, products, and customers.
- Sales, quantities, and prices are key variables for revenue calculations.
- Common beginner mistakes include summing incorrect columns, grouping by the wrong field, or ignoring missing values.
- Forecast evaluation compares predicted values to actual sales using clear business metrics.
- We focus on practical approaches that help stores avoid stockouts and lost revenue.
# Load a real retail sales data example
url = 'https://archive.ics.uci.edu/ml/machine-learning-databases/00502/online_retail_II.xlsx'
df = pd.read_excel(url, sheet_name='Year 2010-2011')
df['InvoiceDate'] = pd.to_datetime(df['InvoiceDate'])
print(df.shape)
print(df.head(3))
# Clean up negative and zero sales transactions
df = df[df['Quantity'] > 0]
df = df[df['Price'] > 0]
print('Cleaned data shape:', df.shape)
# Prepare daily sales for realistic forecasting
daily_sales = df.groupby(df['InvoiceDate'].dt.date)['Quantity'].sum().reset_index(name='ActualSales')
print(daily_sales.head(3))
# Simulate a basic naive sales forecast (last-day prediction)
np.random.seed(42)
daily_sales['ForecastSales'] = daily_sales['ActualSales'].shift(1).bfill()
print(daily_sales.head(5))
Beginner Example 1: Calculate Forecast Error#
- The forecast error shows how far off our forecast was from the actual sales.
- It is calculated as Actual Sales minus Forecast Sales for each day.
- Lower errors mean better accuracy.
# Calculate forecast errors
daily_sales['Error'] = daily_sales['ActualSales'] - daily_sales['ForecastSales']
print(daily_sales[['InvoiceDate','ActualSales','ForecastSales','Error']].head(7))
Beginner Example 2: Calculate MAE (Mean Absolute Error)#
- Mean Absolute Error shows the average mistake size in the forecast, ignoring direction.
- MAE is a common, simple accuracy metric used by retailers to judge forecast reliability.
mae = np.mean(np.abs(daily_sales['Error']))
print('Mean Absolute Error (MAE):', round(mae, 2))
Beginner Example 3: Visualize Actual vs Forecasted Sales#
- Visual comparisons help spot patterns where forecasts systematically over- or under-predict sales.
- Retailers use these charts to quickly identify trouble periods for inventory or promotions.
import matplotlib.pyplot as plt
plt.figure(figsize=(10,5))
plt.plot(daily_sales['InvoiceDate'], daily_sales['ActualSales'], label='Actual Sales')
plt.plot(daily_sales['InvoiceDate'], daily_sales['ForecastSales'], label='Forecast Sales')
plt.xlabel('Date')
plt.ylabel('Sales Quantity')
plt.title('Actual vs Forecasted Daily Sales')
plt.legend()
plt.tight_layout()
plt.show()
Intermediate Example 1: Compute MAPE (Mean Absolute Percentage Error)#
- MAPE measures forecast precision in percentage terms, making it easier to compare across products or stores.
- It is useful for retailers with wide sales ranges.
mape = np.mean(np.abs(daily_sales['Error'] / daily_sales['ActualSales'])) * 100
print('MAPE (%):', round(mape, 2))
Intermediate Example 2: Calculate RMSE (Root Mean Squared Error)#
- RMSE is sensitive to large mistakes, highlighting when forecasts miss by a lot.
- Retailers use RMSE when big errors are especially costly, such as with promotions or perishable goods.
rmse = np.sqrt(np.mean(daily_sales['Error'] ** 2))
print('Root Mean Squared Error (RMSE):', round(rmse, 2))
Intermediate Example 3: Rolling Window Forecasting#
- Businesses update forecasts frequently using rolling averages to smooth out volatile daily patterns.
- This technique is a step up from naive forecasting.
# Create a 7-day rolling mean forecast and compare error
daily_sales['Rolling7Forecast'] = daily_sales['ActualSales'].shift(1).rolling(window=7, min_periods=1).mean()
daily_sales['Rolling7Error'] = daily_sales['ActualSales'] - daily_sales['Rolling7Forecast']
rolling_mae = np.mean(np.abs(daily_sales['Rolling7Error']))
print('7-Day Rolling Forecast MAE:', round(rolling_mae,2))
Advanced Example 1: Segment Accuracy by Country#
- Retail companies may forecast differently in each region or country.
- Measuring errors by segment reveals where the model needs improvement.
# Calculate average forecast error by country using total daily sales
country_sales = df.groupby(['Country', df['InvoiceDate'].dt.date])['Quantity'].sum().reset_index()
country_sales['CountryForecast'] = country_sales.groupby('Country')['Quantity'].shift(1).bfill()
country_sales['CountryError'] = country_sales['Quantity'] - country_sales['CountryForecast']
country_mae = country_sales.groupby('Country')['CountryError'].apply(lambda x: np.mean(np.abs(x))).sort_values(ascending=False)
print(country_mae.head(10))
Advanced Example 2: Category-Level Error Analysis#
- Large retailers track not just total sales, but category performance in accuracy.
- Segmenting error by product line helps inform restocking and promotional strategies.
# Simulate category mapping for demonstration
np.random.seed(42)
df['Category'] = np.random.choice(['Home','Electronics','Clothing','Toys'], df.shape[0])
cat_sales = df.groupby(['Category', df['InvoiceDate'].dt.date])['Quantity'].sum().reset_index()
cat_sales['CatForecast'] = cat_sales.groupby('Category')['Quantity'].shift(1).bfill()
cat_sales['CatError'] = cat_sales['Quantity'] - cat_sales['CatForecast']
cat_mae = cat_sales.groupby('Category')['CatError'].apply(lambda x: np.mean(np.abs(x))).sort_values()
print(cat_mae)
Error Handling Example 1: Handling Missing Actual Sales#
- Retail datasets sometimes have missing or incomplete transactions.
- Ignoring these can distort forecast accuracy calculations.
# Introduce artificial missing values in actuals
temp = daily_sales.copy()
temp.loc[[5,10,15], 'ActualSales'] = np.nan
# Safely recalculate error ignoring missing rows
temp = temp.dropna(subset=['ActualSales'])
temp['Error'] = temp['ActualSales'] - temp['ForecastSales']
print(temp[['InvoiceDate','ActualSales','ForecastSales','Error']].head(17))
Error Handling Example 2: Incorrect Grouping or Aggregation#
- Wrongly grouping sales data can produce misleading errors or performance reports.
- Always check that you group by the correct column (for example, date instead of invoice).
# Grouping by the wrong column: Invoice instead of Date
wrong_group = df.groupby('Invoice')['Quantity'].sum().reset_index(name='InvoiceSum')
print(wrong_group.head(5))
Best Practice: Segment Forecasts by Customer Segment#
- Different customer segments may behave very differentlyoften, high-value and low-frequency customers each need their own forecast.
- Segmenting allows for more precise and useful accuracy measures.
# Identify top customers and calculate forecast MAE for them
top_custs = df['Customer ID'].value_counts().index[:5]
cust_sales = df[df['Customer ID'].isin(top_custs)].groupby(['Customer ID', df['InvoiceDate'].dt.date])['Quantity'].sum().reset_index()
cust_sales['CustForecast'] = cust_sales.groupby('Customer ID')['Quantity'].shift(1).bfill()
cust_sales['CustError'] = cust_sales['Quantity'] - cust_sales['CustForecast']
cust_mae = cust_sales.groupby('Customer ID')['CustError'].apply(lambda x: np.mean(np.abs(x))).sort_values()
print(cust_mae)
Best Practice: Leverage Forecast Accuracy to Improve Inventory#
- Consistently tracking errors allows businesses to adjust safety stock and promotional calendars.
- A good error analysis helps reduce lost sales, waste, and out-of-stock scenarios.
# Demonstrate a simple business action: adjust safety stock for high-error weeks
daily_sales['Week'] = pd.to_datetime(daily_sales['InvoiceDate']).dt.isocalendar().week
weekly_mae = daily_sales.groupby('Week')['Error'].apply(lambda x: np.mean(np.abs(x)))
high_error_weeks = weekly_mae[weekly_mae > weekly_mae.mean()].index.tolist()
print('Weeks to review safety stock:', high_error_weeks)
Advanced Example: Forecast Accuracy for Trend Shifts (Seasonality)#
- Some products or stores experience large swings (for example, holidays, season changes) making error tracking even more important.
- Businesses compare errors before and during holiday periods to improve future forecasts.
# Mark December as Holiday and compare MAE
daily_sales['InvoiceDate'] = pd.to_datetime(daily_sales['InvoiceDate'])
daily_sales['IsHoliday'] = daily_sales['InvoiceDate'].dt.month == 12
holiday_mae = daily_sales.groupby('IsHoliday')['Error'].apply(lambda x: np.mean(np.abs(x)))
print('Non-Holiday MAE:', round(holiday_mae[False],2))
print('Holiday MAE:', round(holiday_mae[True],2))
End-to-End Problem: Find Top Forecast Misses for Critical Review#
- A retail team wants to review days with the largest forecast mistakes to prevent repeating costly errors.
- Identifying the top misses is a simple, actionable analysis for any forecast evaluation meeting.
# List the five days with the largest forecast absolute errors
top_misses = daily_sales.loc[np.abs(daily_sales['Error']).nlargest(5).index]
print(top_misses[['InvoiceDate','ActualSales','ForecastSales','Error']])
Found this useful?
All lessons, notebooks and datasets here are free. If they helped you, a coffee keeps new lessons coming.
Related lessons
16 minLesson 33
Understanding 7 Key Forecast Accuracy Metrics for Evaluating Models in Python
14 minLesson 29
Master Accuracy, Precision & Recall for Evaluating Classification Models in Python
29 minLesson 5
Python Data Analytics #05: Sales Forecasting for Retail Demand in Python
29 minLesson 10