Lesson 7 · Real-World Data Analytics

Python Data Analytics #07: Customer Lifetime Value (CLV) Modelling in Python

Video seven of the hundred-video real-world data analytics series. Predicting real customer value forward, then honestly checking the real prediction…

⬇ Download notebook ⬇ Notebook + data (.zip)Open in Colab ↗

CourseReal-World Data Analytics
Lesson7 of 100
Video29 min
FormatJupyter notebook · 30 code cells
Data1 dataset

pandas NumPy Matplotlib

What you'll learn

Why CLV Needs a Real Calibration and Real Holdout Split
Splitting Real Calibration and Real Holdout Periods
Real Per-Customer Calibration Features
Real Average Order Value per Customer
Real Purchase Frequency per Month
The Real Simple CLV Formula
Real Top Predicted-CLV Customers
Real Total Predicted Portfolio Value

Datasets used in this lesson

Save these next to the notebook. In Google Colab, upload them with the 📁 icon on the left first.

online_retail_clean.csv36.2 MB

📓 Full notebook

Download .ipynb

Data Analytics 100, Video 7: Customer Lifetime Value Modeling#

Video seven of the hundred-video real-world data analytics series.
Predicting real customer value forward, then honestly checking the real prediction against real revenue that actually happened next.
Let's get into it.

Part 1: Why CLV Needs a Real Calibration and Real Holdout Split#

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
clean = pd.read_csv('online_retail_clean.csv', parse_dates=['InvoiceDate'])
clean['CustomerID'] = clean['CustomerID'].astype(int)
clean['InvoiceDate'].min(), clean['InvoiceDate'].max()

(Timestamp('2010-12-01 08:26:00'), Timestamp('2011-12-09 12:50:00'))

Part 2: Splitting Real Calibration and Real Holdout Periods#

cutoff = pd.Timestamp('2011-09-01')
calib = clean[clean['InvoiceDate'] < cutoff]
holdout = clean[clean['InvoiceDate'] >= cutoff]
calib.shape[0], holdout.shape[0]

(223127, 168023)

Part 3: Real Per-Customer Calibration Features#

cust = calib.groupby('CustomerID').agg(TotalRevenue=('Revenue', 'sum'), Orders=('InvoiceNo', 'nunique'), FirstPurchase=('InvoiceDate', 'min'), LastPurchase=('InvoiceDate', 'max'))
cust.shape[0]

Part 4: Real Average Order Value per Customer#

cust['AvgOrderValue'] = cust['TotalRevenue'] / cust['Orders']
cust['AvgOrderValue'].describe().round(2)

count     3314.00
mean       397.76
std       1443.11
min          2.90
25%        167.16
50%        279.09
75%        409.35
max      77183.60
Name: AvgOrderValue, dtype: float64

Part 5: Real Purchase Frequency per Month#

calib_period_days = (cutoff - calib['InvoiceDate'].min()).days
cust['PurchaseFreqPerMonth'] = cust['Orders'] / (calib_period_days / 30)
cust['PurchaseFreqPerMonth'].describe().round(3)

count    3314.000
mean        0.376
std         0.609
min         0.110
25%         0.110
50%         0.220
75%         0.440
max        13.846
Name: PurchaseFreqPerMonth, dtype: float64

Part 6: The Real Simple CLV Formula#

lifespan_months = 12
cust['PredictedCLV'] = cust['AvgOrderValue'] * cust['PurchaseFreqPerMonth'] * lifespan_months
cust['PredictedCLV'].describe().round(2)

count      3314.00
mean       2048.21
std        7807.41
min           3.82
25%         337.48
50%         715.82
75%        1767.17
max      231801.68
Name: PredictedCLV, dtype: float64

Part 7: Real Top Predicted-CLV Customers#

cust.sort_values('PredictedCLV', ascending=False)[['AvgOrderValue', 'PurchaseFreqPerMonth', 'PredictedCLV']].head(10).round(2)

	AvgOrderValue	PurchaseFreqPerMonth	PredictedCLV
CustomerID
14646	4394.57	4.40	231801.68
18102	5020.66	2.86	172137.01
12415	8233.01	1.32	130280.65
17450	3891.38	2.42	112892.76
14156	2417.92	3.85	111596.51
12346	77183.60	0.11	101780.57
14911	666.33	11.32	90503.30
17511	2808.77	2.20	74077.54
13694	1177.22	4.29	60542.82
17949	1562.95	3.19	59770.02

Part 8: Real Total Predicted Portfolio Value#

total_predicted_clv = cust['PredictedCLV'].sum()
print(f'Total real predicted twelve-month CLV across the customer base: {total_predicted_clv:,.0f} GBP')

Total real predicted twelve-month CLV across the customer base: 6,787,762 GBP

Part 9: Real Actual Revenue During the Real Holdout Window#

actual_holdout_rev = holdout.groupby('CustomerID')['Revenue'].sum()
cust['ActualHoldoutRevenue'] = cust.index.map(actual_holdout_rev).fillna(0)
(cust['ActualHoldoutRevenue'] > 0).mean().round(3)

np.float64(0.588)

Part 10: Scaling the Real Prediction to the Real Holdout Horizon#

holdout_months = (clean['InvoiceDate'].max() - cutoff).days / 30
cust['PredictedHoldoutRevenue'] = cust['AvgOrderValue'] * cust['PurchaseFreqPerMonth'] * holdout_months
round(holdout_months, 2)

3.3

Part 11: Real Validation, Correlation#

correlation = cust[['PredictedHoldoutRevenue', 'ActualHoldoutRevenue']].corr().iloc[0, 1]
round(correlation, 3)

np.float64(0.68)

Part 12: Real Validation, Mean Absolute Error#

holdout_mae = (cust['PredictedHoldoutRevenue'] - cust['ActualHoldoutRevenue']).abs().mean()
round(holdout_mae, 2)

np.float64(624.8)

Part 13: Visualizing Real Predicted vs Real Actual#

plt.figure(figsize=(8, 8))
plt.scatter(cust['PredictedHoldoutRevenue'], cust['ActualHoldoutRevenue'], alpha=0.3, s=15, color='teal')
max_val = max(cust['PredictedHoldoutRevenue'].quantile(0.99), cust['ActualHoldoutRevenue'].quantile(0.99))
plt.plot([0, max_val], [0, max_val], color='red', linestyle='--', label='Real perfect prediction line')
plt.xlim(0, max_val)
plt.ylim(0, max_val)
plt.xlabel('Real Predicted Holdout Revenue')
plt.ylabel('Real Actual Holdout Revenue')
plt.title('Real Predicted vs Real Actual Customer Revenue')
plt.legend()
plt.tight_layout()
plt.savefig('clv_predicted_vs_actual.png', dpi=120)
plt.close()

Part 14: Rebuilding Real RFM Segments for This Same Population#

snapshot_date = calib['InvoiceDate'].max() + pd.Timedelta(days=1)
recency = (snapshot_date - cust['LastPurchase']).dt.days
cust['R_score'] = pd.qcut(recency, 4, labels=[4, 3, 2, 1], duplicates='drop').astype(int)
cust['R_score'].value_counts().sort_index()

R_score
1    816
2    841
3    807
4    850
Name: count, dtype: int64

Part 15: Real Average Predicted CLV by Recency Score#

cust.groupby('R_score')['PredictedCLV'].mean().round(2).sort_index(ascending=False)

R_score
4    4644.50
3    1768.44
2    1008.67
1     691.81
Name: PredictedCLV, dtype: float64

Part 16: Real Average Predicted CLV by Order Frequency Tier#

cust['F_score'] = pd.qcut(cust['Orders'].rank(method='first'), 4, labels=[1, 2, 3, 4]).astype(int)
cust.groupby('F_score')['PredictedCLV'].mean().round(2).sort_index(ascending=False)

F_score
4    5608.89
3    1340.94
2     622.85
1     617.58
Name: PredictedCLV, dtype: float64

Part 17: Real High-Predicted, Zero-Actual Churn Risk Flags#

high_predicted = cust[cust['PredictedCLV'] > cust['PredictedCLV'].quantile(0.9)]
churn_risk = high_predicted[high_predicted['ActualHoldoutRevenue'] == 0]
churn_risk.shape[0], high_predicted.shape[0]

(33, 332)

Part 18: Real Lost Value from Those Churn-Risk Customers#

lost_predicted_value = churn_risk['PredictedHoldoutRevenue'].sum()
print(f'Real predicted revenue left on the table from likely-churned high-value customers: {lost_predicted_value:,.0f} GBP')

Real predicted revenue left on the table from likely-churned high-value customers: 107,562 GBP

Part 19: Real Sensitivity to the Lifespan Assumption#

for months in [6, 12, 18, 24]:
    total = (cust['AvgOrderValue'] * cust['PurchaseFreqPerMonth'] * months).sum()
    print(f'{months}-month lifespan assumption: {total:,.0f} GBP total real predicted CLV')

6-month lifespan assumption: 3,393,881 GBP total real predicted CLV
12-month lifespan assumption: 6,787,762 GBP total real predicted CLV
18-month lifespan assumption: 10,181,643 GBP total real predicted CLV
24-month lifespan assumption: 13,575,524 GBP total real predicted CLV

Part 20: Real One-Time Buyers vs Real Predicted CLV#

one_timers = cust[cust['Orders'] == 1]
one_timers['PredictedCLV'].mean().round(2), cust[cust['Orders'] > 1]['PredictedCLV'].mean().round(2)

(np.float64(531.39), np.float64(3129.19))

Part 21: Real Sanity Check on the Formula#

sample_customer = cust.sort_values('PredictedCLV', ascending=False).index[0]
manual_clv = cust.loc[sample_customer, 'AvgOrderValue'] * cust.loc[sample_customer, 'PurchaseFreqPerMonth'] * lifespan_months
abs(manual_clv - cust.loc[sample_customer, 'PredictedCLV']) < 0.01

np.True_

Part 22: Saving the Real CLV Table#

output_cols = ['AvgOrderValue', 'PurchaseFreqPerMonth', 'PredictedCLV', 'ActualHoldoutRevenue', 'R_score', 'F_score']
cust[output_cols].round(2).to_csv('customer_lifetime_value.csv')
reloaded = pd.read_csv('customer_lifetime_value.csv')
reloaded.shape[0] == cust.shape[0]

True

Part 23: Real Final Business Summary#

print(f'Predicted CLV for {len(cust)} real customers; model correlation with real actual holdout spend was {round(correlation, 2)}.')

Predicted CLV for 3314 real customers; model correlation with real actual holdout spend was 0.68.

Part 24: Visualizing the Real Predicted CLV Distribution#

plt.figure(figsize=(9, 5))
plt.hist(cust['PredictedCLV'].clip(upper=cust['PredictedCLV'].quantile(0.95)), bins=40, color='indigo', edgecolor='white')
plt.xlabel('Real Predicted Twelve-Month CLV (GBP)')
plt.ylabel('Real Number of Customers')
plt.title('Real Distribution of Predicted Customer Lifetime Value')
plt.tight_layout()
plt.savefig('clv_distribution.png', dpi=120)
plt.close()

Part 25: Real Average Predicted CLV by Country#

country_lookup = calib.groupby('CustomerID')['Country'].agg(lambda s: s.mode().iloc[0])
cust['Country'] = cust.index.map(country_lookup)
cust.groupby('Country')['PredictedCLV'].mean().sort_values(ascending=False).head(8).round(2)

Country
EIRE           69252.85
Netherlands    29556.66
Australia      17955.57
Singapore      10709.18
Japan           5911.25
Israel          4670.55
Sweden          4536.31
Iceland         3680.25
Name: PredictedCLV, dtype: float64

Part 26: Real Mean vs Real Median CLV#

cust['PredictedCLV'].mean().round(2), cust['PredictedCLV'].median().round(2)

(np.float64(2048.21), np.float64(715.82))

Part 27: Does the Real Full Formula Beat a Real Naive Baseline#

naive_correlation = cust[['AvgOrderValue', 'ActualHoldoutRevenue']].corr().iloc[0, 1]
round(naive_correlation, 3), round(correlation, 3)

(np.float64(0.091), np.float64(0.68))

Part 28: Real Total Actual Revenue Cross-Check#

total_actual_from_cust = cust['ActualHoldoutRevenue'].sum()
total_actual_from_holdout = holdout[holdout['CustomerID'].isin(cust.index)]['Revenue'].sum()
abs(total_actual_from_cust - total_actual_from_holdout) < 1

np.True_

Part 29: Real High-Value Customer Count by Threshold#

for threshold in [500, 1000, 2000, 5000]:
    n_above = (cust['PredictedCLV'] > threshold).sum()
    print(f'{n_above} real customers predicted above {threshold} GBP twelve-month CLV')

2029 real customers predicted above 500 GBP twelve-month CLV
1316 real customers predicted above 1000 GBP twelve-month CLV
719 real customers predicted above 2000 GBP twelve-month CLV
217 real customers predicted above 5000 GBP twelve-month CLV

Part 30: Real Number of Customers With Any Predicted Value#

(cust['PredictedCLV'] > 0).sum()

np.int64(3314)

Wrap-Up: What You Learned#

A real CLV estimate is genuinely just average order value, times real purchase frequency, times an assumed real customer lifespan.
Any real CLV model needs a real calibration and real holdout split; predicting the real future and then checking it is the whole point.
A real positive correlation between predicted and real actual holdout revenue is what actually earns the model any trust.
Segmenting predicted CLV by real recency and real frequency shows exactly which customer types are genuinely worth investing in.
Customers with real high predicted value but real zero actual return are the real highest-value churn-risk flags a team can act on.
Next video: real product category performance and ABC analysis, ranking what this business actually sells best.

Found this useful?

All lessons, notebooks and datasets here are free. If they helped you, a coffee keeps new lessons coming.

☕ Buy me a coffee Other ways to help