Mathew K Analytics

Lesson 10 · Real-World Data Analytics

End-to-End Retail Analytics Project in Python (Portfolio Capstone) | Data Analytics #10

A complete data analyst portfolio project: we combine data cleaning, RFM segmentation, market basket analysis, cohort retention, sales forecasting, price…

What you'll learn

Datasets used in this lesson

Save these next to the notebook. In Google Colab, upload them with the 📁 icon on the left first.

📓 Full notebook

Download .ipynb

Data Analytics 100, Video 10: Capstone, End-to-End Retail Analytics Report#

  • Video ten of the hundred-video real-world data analytics series, and the real capstone for Domain 1.
  • Pulling every technique from this real retail series into one real consolidated end-to-end report.
  • Let's get into it.

This capstone is for learners who have worked through the first nine lessons of the series, or who want to see how a full retail analysis fits together in one place. The business question is the one a senior manager actually asks: across customers, products and time, what is happening in this business, and what should we do next?

You will use online_retail_clean.csv, the cleaned online retail transactions built in the first lesson, with CustomerID, InvoiceNo, InvoiceDate, StockCode, Description, Quantity, UnitPrice, Country and Revenue. Every headline number is recomputed fresh from this one file rather than copied from earlier notebooks, so the report is consistent and repeatable.

In this lesson you will:

  • recompute the key results from RFM segmentation, market basket analysis, cohort retention, forecasting, price elasticity, customer lifetime value and ABC analysis
  • combine four charts into a single dashboard figure
  • write and save an executive summary and a metrics table
  • turn the numbers into one clear business recommendation

You should be comfortable with pandas groupby, pd.qcut, merge and basic matplotlib. Each recap is brief, so revisit the matching earlier lesson if a technique feels unfamiliar.

Part 1: One Real Dataset, Nine Real Techniques#

A capstone report is only trustworthy if every figure comes from the same source data. This step loads the cleaned file once, parses InvoiceDate as dates and converts CustomerID to whole numbers with astype(int), which works because the cleaned data has no missing customer IDs. The output (391150, 9) confirms the familiar 391,150 rows and 9 columns. Every technique that follows reads from this single clean DataFrame, so any later change to cleaning flows through the whole report.

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
clean = pd.read_csv('online_retail_clean.csv', parse_dates=['InvoiceDate'])
clean['CustomerID'] = clean['CustomerID'].astype(int)
clean.shape
(391150, 9)

Part 2: Real Data Quality Recap#

Before presenting results, state the scale of what you analysed so readers can put every other number in context. The code counts distinct customers and invoices with nunique() and sums Revenue, then prints them with :,.0f formatting for thousands separators. The output shows 4,334 customers, 18,402 invoices and about 8,737,228 GBP of total revenue. These three figures become the denominators that later percentages and shares are judged against.

n_customers = clean['CustomerID'].nunique()
n_invoices = clean['InvoiceNo'].nunique()
total_revenue = clean['Revenue'].sum()
print(f'{n_customers} real customers, {n_invoices} real invoices, {total_revenue:,.0f} GBP total real revenue.')
4334 real customers, 18402 real invoices, 8,737,228 GBP total real revenue.

Part 3: Real RFM Segmentation Recap#

RFM stands for recency (how recently a customer bought), frequency (how often) and monetary value (how much they spent). The code computes all three per customer with named aggregation, then uses pd.qcut to score recency and frequency from 1 to 4. Ranking frequency with .rank(method='first') first avoids errors from tied values. Champions are customers scoring at least 3 on both. The output shows 1,524 Champions worth about 6,469,790 GBP, roughly 74% of total revenue from about a third of customers.

snapshot_date = clean['InvoiceDate'].max() + pd.Timedelta(days=1)
rfm = clean.groupby('CustomerID').agg(Recency=('InvoiceDate', lambda s: (snapshot_date - s.max()).days), Frequency=('InvoiceNo', 'nunique'), Monetary=('Revenue', 'sum'))
rfm['R_score'] = pd.qcut(rfm['Recency'], 4, labels=[4, 3, 2, 1]).astype(int)
rfm['F_score'] = pd.qcut(rfm['Frequency'].rank(method='first'), 4, labels=[1, 2, 3, 4]).astype(int)
champions = rfm[(rfm['R_score'] >= 3) & (rfm['F_score'] >= 3)]
print(f'{len(champions)} real Champion customers, worth {champions["Monetary"].sum():,.0f} GBP in real total historical revenue.')
1524 real Champion customers, worth 6,469,790 GBP in real total historical revenue.

Part 4: Real Market Basket Recap#

Market basket analysis looks for products that are bought together, which helps with cross-selling and store layout. To keep things manageable, the code uses UK orders only and the 30 products appearing on the most invoices. It then groups by invoice and uses .apply(set) to collect each basket's products, keeping baskets with at least two of them. The output shows 6,559 multi-item baskets. Using set removes duplicate lines of the same product within an invoice, which matters when counting pairs.

uk = clean[clean['Country'] == 'United Kingdom']
top_products = uk.groupby('Description')['InvoiceNo'].nunique().sort_values(ascending=False).head(30)
baskets = uk[uk['Description'].isin(top_products.index)].groupby('InvoiceNo')['Description'].apply(set)
baskets = baskets[baskets.apply(len) >= 2]
print(f'{len(baskets)} real multi-item baskets found among the real top thirty products.')
6559 real multi-item baskets found among the real top thirty products.

Part 5: Real Cohort Retention Recap#

Cohort retention tracks what share of customers who first bought in a given month come back in later months. The code truncates each date to its month with astype('datetime64[M]'), finds each customer's first month, merges it back, and counts months since that first purchase as CohortIndex. A pivot of distinct customers is then divided by each cohort's starting size with .divide(..., axis=0). The output shows average month-one retention of 20.4%: about one in five new customers buys again the following month.

clean['InvoiceMonth'] = clean['InvoiceDate'].values.astype('datetime64[M]')
first_purchase = clean.groupby('CustomerID')['InvoiceMonth'].min()
merged = clean.merge(first_purchase.rename('CohortMonth'), on='CustomerID')
merged['CohortIndex'] = (merged['InvoiceMonth'].dt.year - merged['CohortMonth'].dt.year) * 12 + (merged['InvoiceMonth'].dt.month - merged['CohortMonth'].dt.month)
cohort_pivot = merged.groupby(['CohortMonth', 'CohortIndex'])['CustomerID'].nunique().unstack()
retention = cohort_pivot.divide(cohort_pivot[0], axis=0)
avg_month1_retention = retention[1].mean()
print(f'Average real month-one retention across all cohorts: {round(avg_month1_retention * 100, 1)}%.')
Average real month-one retention across all cohorts: 20.4%.

Part 6: Real Sales Forecast Recap#

A short-term revenue forecast helps with staffing and stock planning. The code sums revenue by day, uses resample('W') to total it by week, and drops the first and last weeks with .iloc[1:-1] because they are likely incomplete. It then fits ExponentialSmoothing from statsmodels with an additive trend and no seasonality, a method that weights recent weeks more heavily, and forecasts one week ahead. The output is about 270,317 GBP. With only about a year of data, the model cannot learn yearly seasonality, so treat this as a rough guide.

from statsmodels.tsa.holtwinters import ExponentialSmoothing
daily = clean.groupby(clean['InvoiceDate'].dt.date)['Revenue'].sum()
daily.index = pd.to_datetime(daily.index)
weekly = daily.resample('W').sum().iloc[1:-1]
forecast_model = ExponentialSmoothing(weekly, trend='add', seasonal=None).fit()
next_week_forecast = forecast_model.forecast(1).iloc[0]
print(f'Next week real forecasted revenue: {next_week_forecast:,.0f} GBP.')
Next week real forecasted revenue: 270,317 GBP.

Part 7: Real Price Elasticity Recap#

Price elasticity measures how much quantity sold changes when price changes by 1%. The code repeats the earlier method: keep products with at least four distinct prices and over 500 units sold, sum quantity at each price, fit a straight line to the logged values with np.polyfit, and treat the slope as elasticity. Products with a slope below -1 are counted as elastic, meaning demand responds strongly to price. The output shows 303 of 373 qualifying products are elastic, matching the price and discount lesson.

qty_by_sc = clean.groupby('StockCode')['Quantity'].sum()
price_var_by_sc = clean.groupby('StockCode')['UnitPrice'].nunique()
candidates = price_var_by_sc[(price_var_by_sc >= 4) & (qty_by_sc > 500)].index
elastic_count = 0
for sc in candidates:
    bp = clean[clean['StockCode'] == sc].groupby('UnitPrice')['Quantity'].sum()
    bp = bp[bp > 0]
    if len(bp) < 4 or bp.index.to_series().std() == 0:
        continue
    e, _ = np.polyfit(np.log(bp.index.values.astype(float)), np.log(bp.values.astype(float)), 1)
    elastic_count += 1 if e < -1 else 0
print(f'{elastic_count} of {len(candidates)} real qualifying products are genuinely elastic, good discounting candidates.')
303 of 373 real qualifying products are genuinely elastic, good discounting candidates.

Part 8: Real Customer Lifetime Value Recap#

Customer lifetime value (CLV) estimates how much revenue a customer will bring in over a period. The code uses the simple formula from the CLV lesson: average order value times orders per month times 12 months. Here it uses the full date range, so there is no holdout test this time. The output gives a total predicted 12-month CLV of about 8,432,713 GBP. That is close to the historical total because the data covers just over a year, so the formula mostly rescales past revenue. It assumes customers keep buying at their past rate.

cust_orders = clean.groupby('CustomerID').agg(TotalRevenue=('Revenue', 'sum'), Orders=('InvoiceNo', 'nunique'))
cust_orders['AvgOrderValue'] = cust_orders['TotalRevenue'] / cust_orders['Orders']
total_days = (clean['InvoiceDate'].max() - clean['InvoiceDate'].min()).days
cust_orders['FreqPerMonth'] = cust_orders['Orders'] / (total_days / 30)
cust_orders['PredictedCLV'] = cust_orders['AvgOrderValue'] * cust_orders['FreqPerMonth'] * 12
total_predicted_clv = cust_orders['PredictedCLV'].sum()
print(f'Total real predicted twelve-month CLV across the customer base: {total_predicted_clv:,.0f} GBP.')
Total real predicted twelve-month CLV across the customer base: 8,432,713 GBP.

Part 9: Real ABC Analysis Recap#

ABC analysis ranks products by revenue to show where it is concentrated. The code sums revenue per product, sorts from highest to lowest, builds a running cumulative percentage with .cumsum(), and counts products at or below 80%, which form tier A. The output shows 21.4% of products drive 80% of revenue, close to the classic 80/20 Pareto rule. For a business, this means stock availability and pricing on a relatively small set of products deserve the most attention.

product_revenue = clean.groupby('StockCode')['Revenue'].sum().sort_values(ascending=False)
cumulative_pct = product_revenue.cumsum() / product_revenue.sum() * 100
n_tier_a = (cumulative_pct <= 80).sum()
pct_tier_a = n_tier_a / len(product_revenue) * 100
print(f'{round(pct_tier_a, 1)}% of products drive 80% of total real revenue.')
21.4% of products drive 80% of total real revenue.

Part 10: Building the Real Consolidated Dashboard#

Stakeholders prefer one picture to many separate charts. plt.subplots(2, 2) creates a grid of four panels, and each is drawn through its own axes[row, col]: weekly revenue history, the average retention curve across cohorts, the revenue Pareto curve with a dashed 80% line, and a bar chart of Champions against everyone else. Open capstone_dashboard.png and check that each panel tells one clear story, for example the rise in weekly revenue towards the end of the year and the steep early climb of the Pareto curve.

fig, axes = plt.subplots(2, 2, figsize=(14, 10))
axes[0, 0].plot(weekly.index, weekly.values, color='navy')
axes[0, 0].set_title('Real Weekly Revenue History')
axes[0, 1].plot(retention.columns, retention.mean(), marker='o', color='seagreen')
axes[0, 1].set_title('Real Average Retention Curve')
rank = np.arange(1, len(cumulative_pct) + 1)
axes[1, 0].plot(rank, cumulative_pct.values, color='crimson')
axes[1, 0].axhline(80, color='black', linestyle='--')
axes[1, 0].set_title('Real Revenue Pareto Curve')
segment_sizes = pd.Series({'Champions': len(champions), 'Everyone Else': n_customers - len(champions)})
axes[1, 1].bar(segment_sizes.index, segment_sizes.values, color=['gold', 'gray'])
axes[1, 1].set_title('Real Champions vs Real Everyone Else')
plt.tight_layout()
plt.savefig('capstone_dashboard.png', dpi=120)
plt.close()

Part 11: Writing the Real Executive Summary#

An executive summary turns the numbers into short, readable statements. The code builds a Python list and appends one line per finding with f-strings, reusing values computed earlier: dataset size, Champions, retention, the forecast, elastic products, CLV and the tier A share. There is no output because nothing is printed yet. Keeping the summary as a list of variables rather than typed-in numbers means it updates automatically if the data or cleaning ever changes.

summary_lines = []
summary_lines.append('RETAIL ANALYTICS CAPSTONE, EXECUTIVE SUMMARY')
summary_lines.append(f'Dataset: {n_customers} real customers, {n_invoices} real invoices, {total_revenue:,.0f} GBP total revenue')
summary_lines.append(f'Champions segment: {len(champions)} customers, {champions["Monetary"].sum():,.0f} GBP historical revenue')
summary_lines.append(f'Average month-1 retention across cohorts: {round(avg_month1_retention * 100, 1)}%')
summary_lines.append(f'Next week forecasted revenue: {next_week_forecast:,.0f} GBP')
summary_lines.append(f'{elastic_count} of {len(candidates)} qualifying products are price-elastic')
summary_lines.append(f'Total predicted 12-month customer lifetime value: {total_predicted_clv:,.0f} GBP')
summary_lines.append(f'{round(pct_tier_a, 1)}% of products (tier A) drive 80% of total revenue')

Part 12: Saving the Real Executive Summary#

A summary only helps if it leaves the notebook. The code opens capstone_executive_summary.txt in write mode inside a with block, which closes the file automatically, and writes each line followed by chr(10), the newline character. The output confirms the summary was saved. A plain text file is easy to paste into an email or a slide, and anyone can open it without Python.

with open('capstone_executive_summary.txt', 'w') as f:
    for line in summary_lines:
        f.write(line + chr(10))
print('Real executive summary saved.')
Real executive summary saved.

Part 13: Real Reload Verification#

Checking a saved file catches silent problems such as an empty write or the wrong folder. The code reopens the summary, splits it into lines with .splitlines(), and compares the line count with the original list. The output True confirms all lines were written. It is a quick, cheap habit worth repeating for every file a report produces.

with open('capstone_executive_summary.txt') as f:
    reloaded_lines = f.read().splitlines()
len(reloaded_lines) == len(summary_lines)
True

Part 14: Real Consolidated Metrics Table#

A metrics table gives the same headline figures in a structured form that can be filtered, charted or loaded into another tool. The code builds a two-column DataFrame from a dictionary of lists. The output lists 4,334 customers, total revenue of about 8,737,228 GBP, 1,524 Champions, 20.4% month-one retention, a forecast of about 270,317 GBP, 303 elastic products, predicted CLV of about 8,432,713 GBP and 21.4% of products in tier A. Counts appear with .0 because pandas stores the mixed Value column as floats.

metrics = pd.DataFrame({
    'Metric': ['Customers', 'Total Revenue', 'Champions', 'Avg Month1 Retention %', 'Next Week Forecast', 'Elastic Products', 'Total Predicted CLV', 'Tier A Product %'],
    'Value': [n_customers, round(total_revenue, 0), len(champions), round(avg_month1_retention * 100, 1), round(next_week_forecast, 0), elastic_count, round(total_predicted_clv, 0), round(pct_tier_a, 1)]
})
metrics
Metric Value
0 Customers 4334.0
1 Total Revenue 8737228.0
2 Champions 1524.0
3 Avg Month1 Retention % 20.4
4 Next Week Forecast 270317.0
5 Elastic Products 303.0
6 Total Predicted CLV 8432713.0
7 Tier A Product % 21.4

Part 15: Saving the Real Metrics Table#

Saving the metrics table lets others track these figures over time, for example by comparing with next quarter's run. The code writes capstone_metrics_table.csv with index=False so the row numbers are not saved as an extra column, then reloads it and compares row counts. The output True confirms all eight metrics were saved.

metrics.to_csv('capstone_metrics_table.csv', index=False)
reloaded_metrics = pd.read_csv('capstone_metrics_table.csv')
reloaded_metrics.shape[0] == metrics.shape[0]
True

Part 16: Real Top-Line Business Recommendation#

The most valuable output of an analysis is a specific action, not a list of numbers. The code builds one recommendation sentence from the computed values and prints it. The output advises focusing retention spend on the 1,524 Champions, worth about 6,469,790 GBP, and using the 303 elastic products as discount levers to win back other customers. Note the reasoning: protect the customers who bring in most of the revenue, and discount only where demand actually responds to price, so margin is not given away.

recommendation = f'Focus retention spend on the {len(champions)} real Champions worth {champions["Monetary"].sum():,.0f} GBP, while using the {elastic_count} genuinely elastic products identified as safe discount levers to win back the segments below them.'
print(recommendation)
Focus retention spend on the 1524 real Champions worth 6,469,790 GBP, while using the 303 genuinely elastic products identified as safe discount levers to win back the segments below them.

Part 17: Real Domain Recap#

This step simply marks the end of the first domain of the series. The code prints a confirmation that Domain 1, Retail and E-Commerce Analytics, is complete across 10 lessons, as shown in the output. It is a good moment to look back at the techniques recapped above and note which ones you would want to practise further before moving on to new data in the next domain.

print('Domain 1, Retail and E-Commerce Analytics: 10 real lessons complete.')
Domain 1, Retail and E-Commerce Analytics: 10 real lessons complete.

Part 4b: Real Top Association Rule#

Counting baskets is a start, but cross-selling needs to know which products go together. The code loops over each basket and uses itertools.combinations to list every pair of products, sorted so that A with B and B with A count as the same pair, and tallies them in a Counter. max(..., key=pair_counts.get) finds the most common pair. The output shows two jumbo bags, pink polka dot and red retrospot (names trimmed to 20 characters), bought together in 506 baskets. That is a natural bundle or recommendation candidate.

from itertools import combinations
from collections import Counter
pair_counts = Counter()
for basket in baskets:
    for a, b in combinations(sorted(basket), 2):
        pair_counts[(a, b)] = pair_counts.get((a, b), 0) + 1
top_pair = max(pair_counts, key=pair_counts.get)
print(f'Most frequent real pairing: {top_pair[0][:20]} + {top_pair[1][:20]}, seen together in {pair_counts[top_pair]} real baskets.')
Most frequent real pairing: JUMBO BAG PINK POLKA + JUMBO BAG RED RETROS, seen together in 506 real baskets.

Part 6b: Real Forecast Residual Sanity Check#

Residuals are the differences between actual values and the model's fitted values. If a model is unbiased, they should average close to zero. The code subtracts forecast_model.fittedvalues from the weekly revenue and takes the mean. The output is about 7,318 GBP, meaning actual weeks came in above the fitted values on average. Compared with a forecast of about 270,317 GBP that is a modest gap, but it hints the model slightly under-predicts, likely because it cannot capture the strong end-of-year rise.

residuals = weekly - forecast_model.fittedvalues
round(residuals.mean(), 1)
np.float64(7317.9)

Part 9b: Real Tier B and Tier C Breakdown#

The ABC recap above reported only tier A, so this step completes the picture. The code counts products with cumulative share above 80% and up to 95% as tier B, and above 95% as tier C, combining conditions with &. The output shows 783 products in tier A, 920 in B and 1,956 in C. More than half the catalogue sits in tier C, sharing just 5% of revenue, which is worth raising in any discussion of range or stock reduction.

n_tier_b = ((cumulative_pct > 80) & (cumulative_pct <= 95)).sum()
n_tier_c = (cumulative_pct > 95).sum()
print(f'Tier A: {n_tier_a} products, Tier B: {n_tier_b} products, Tier C: {n_tier_c} products.')
Tier A: 783 products, Tier B: 920 products, Tier C: 1956 products.

Part 9c: Real Country Revenue Breakdown#

Knowing which markets drive revenue helps decide where to focus growth. The code sums revenue by country and shows the top three. The output shows the United Kingdom far ahead at about 7,242,855 GBP, roughly 83% of total revenue, followed by the Netherlands at about 283,889 and EIRE at about 257,013. Because the UK dominates so heavily, results for the whole business largely reflect UK customers, which is worth stating when you present them.

clean.groupby('Country')['Revenue'].sum().sort_values(ascending=False).head(3).round(0)
Country
United Kingdom    7242855.0
Netherlands        283889.0
EIRE               257013.0
Name: Revenue, dtype: float64

Part 9d: Real Overall Average Order Value#

Average order value (AOV) is the average revenue per invoice. The code first sums revenue per invoice, then takes the mean of those totals. The output is about 474.80 GBP per order. Doing the two steps in this order matters: taking .mean() of Revenue directly would give the average line item, not the average order, which is a common mistake. Since this shop serves many wholesale buyers, a large AOV like this is not surprising.

overall_aov = clean.groupby('InvoiceNo')['Revenue'].sum().mean()
round(overall_aov, 2)
np.float64(474.8)

Part 9e: Real Busiest Month by Revenue#

Knowing the peak month helps with stock and staffing plans. The code converts dates to monthly periods with .dt.to_period('M'), sums revenue per month, and idxmax() returns the month with the largest total. The output shows November 2011 as the busiest month, with about 1,136,534 GBP. That fits a gift-oriented retailer stocking up shops ahead of Christmas. December 2011 is only partly covered by the data, so do not read its lower total as a real slowdown.

monthly_revenue = clean.groupby(clean['InvoiceDate'].dt.to_period('M'))['Revenue'].sum()
busiest_month = monthly_revenue.idxmax()
print(f'Busiest real month: {busiest_month}, with {monthly_revenue[busiest_month]:,.0f} GBP in real revenue.')
Busiest real month: 2011-11, with 1,136,534 GBP in real revenue.

Part 9f: Real Reconciliation Check#

A reconciliation check confirms that aggregating to customer level did not lose or double-count revenue. The code compares total Monetary value across all RFM customers with the overall revenue total, allowing a difference of under 1 GBP for rounding. The output True confirms the two match. If this ever fails, look for customers dropped by a filter or duplicated by a merge.

abs(rfm['Monetary'].sum() - total_revenue) < 1
np.True_

Part 9g: Real Top 5 Customers by Predicted CLV#

Seeing the top customers by predicted CLV puts names to the value. The code sorts customers by PredictedCLV and shows the top five with their order value and frequency. Customer 14646 leads with about 269,409 GBP. Customer 16446 stands out: an average order value of about 84,236 GBP but only 0.16 orders a month, which points to one or two very large orders rather than a buying habit. A forecast built on such a customer is much less reliable than one for 14911, who orders about 15.92 times a month.

cust_orders.sort_values('PredictedCLV', ascending=False).head(5)[['AvgOrderValue', 'FreqPerMonth', 'PredictedCLV']].round(2)
AvgOrderValue FreqPerMonth PredictedCLV
CustomerID
14646 3876.92 5.79 269409.35
18102 4327.62 4.83 250607.58
17450 4225.89 3.70 187615.78
16446 84236.25 0.16 162600.80
14911 687.69 15.92 131416.24

Part 9h: Real One-Time vs Repeat Customer Split#

The share of customers who come back at least once is a simple measure of loyalty. The code checks whether each customer has more than one distinct invoice and takes .mean() of the True/False result, which gives the share of True values. The output shows 65.3% of customers placed more than one order over the period, so roughly a third bought only once. Those one-time buyers are an obvious target for a second-purchase campaign.

repeat_rate = (cust_orders['Orders'] > 1).mean()
round(repeat_rate * 100, 1)
np.float64(65.3)

Part 9i: Real Total Products in Catalog#

The size of the catalogue gives context for the product-level findings. The code counts distinct StockCode values with nunique(). The output shows 3,659 products, the same total split into the A, B and C tiers earlier (783 plus 920 plus 1,956). Checking that these numbers add up is a quick consistency test across sections of the report.

clean['StockCode'].nunique()
3659

Part 9j: Real Cancellation Rate Sanity Check#

Cancelled invoices, marked with a C at the start of the invoice number, would distort every revenue figure if they slipped back in. The code converts InvoiceNo to text with astype(str), flags numbers starting with C using .str.startswith('C'), and checks the count is zero. The output True confirms the cleaned data contains no cancellations, so the cleaning from the first lesson held up. Re-running checks like this in a final report guards against using the wrong input file.

clean['InvoiceNo'].astype(str).str.startswith('C').sum() == 0
np.True_

Part 9k: Real Median Customer Revenue#

The median shows what a typical customer is worth, without the pull of a few very large accounts. The code takes the median of each customer's total revenue. The output is about 662.56 GBP. Compare that with the average, total revenue divided by 4,334 customers, which works out at roughly 2,016 GBP: the mean is about three times the median. That gap confirms revenue is heavily concentrated in a minority of customers, which supports the focus on Champions.

cust_orders['TotalRevenue'].median().round(2)
np.float64(662.56)

Part 9l: Real Domain 1 Lesson Count Confirmation#

The final step confirms the scope of the domain the capstone covers. The code builds a list of the 10 Domain 1 lesson names and counts them with len. The output confirms 10 lessons. Keep this list handy as a checklist: each name is a technique you can now apply to a new dataset, which is exactly what the next domain will ask you to do.

domain_1_lessons = ['Acquiring & Auditing', 'RFM Segmentation', 'Market Basket', 'Cohort Retention', 'Sales Forecasting', 'Price Elasticity', 'Customer Lifetime Value', 'ABC Analysis', 'Scraping Prices', 'Capstone']
len(domain_1_lessons)
10

Wrap-Up: What You Learned#

  • A real capstone genuinely recomputes every prior headline number fresh, straight from the real same source data, rather than trusting cached results blindly.
  • RFM segmentation, market basket analysis, cohort retention, forecasting, price elasticity, CLV, and ABC analysis all draw on the exact real same cleaned dataset.
  • A real consolidated dashboard, four charts in one figure, is exactly the kind of artifact a real stakeholder actually wants to see.
  • A real written executive summary translates every real computed number into a real plain-language business narrative.
  • The real most valuable output of any analysis is a real specific, actionable recommendation, not just a pile of real numbers.
  • Next up: Domain 2, Finance and Stock Market Analytics, ten more real lessons starting with acquiring real historical stock price data.

Practice on your own

  1. Add a fifth panel to the dashboard by switching to plt.subplots(2, 3) and plotting monthly revenue from monthly_revenue, then save it under a new file name.
  2. Redefine Champions as customers with R_score of 4 and F_score of 4 only, and compare the new count and their share of total revenue with 1,524 customers and about 6,469,790 GBP.
  3. Refit the forecast with ExponentialSmoothing(weekly, trend=None) and compare the next-week forecast and the mean residual with the trend model's 270,317 GBP and 7,317.9.

The next lesson starts Domain 2, Finance and Stock Market Analytics, beginning with acquiring historical stock price data.

🧠 Test yourself

Pick an answer to check it. Your best score is saved in this browser.

  1. Why does the capstone recompute every headline number from online_retail_clean.csv instead of reusing earlier results?

  2. In the capstone, how is the Champions group defined?

    champions = rfm[(rfm['R_score'] >= 3) & (rfm['F_score'] >= 3)]
    
  3. What does n_tier_a = (cumulative_pct <= 80).sum() followed by n_tier_a / len(product_revenue) * 100 tell you?

  4. How does the capstone produce next week's revenue forecast?

  5. What is the purpose of the check abs(rfm['Monetary'].sum() - total_revenue) < 1?

💻 Practice exercises

Try each one in the notebook or in your own editor before opening the solution.

  1. Create a clean transactions DataFrame with 15 rows (InvoiceNo, CustomerID, InvoiceDate, Quantity, UnitPrice, Country). Compute Revenue and print: number of customers, invoices, total revenue, average order value, busiest month by revenue, and the top 2 countries by revenue.

    💡 Hint

    Group by InvoiceNo for order values, and by dt.to_period('M') for months.

    ✅ Show solution
    import pandas as pd
    
    clean = pd.DataFrame({
        'InvoiceNo': [1,1,2,3,3,4,5,6,6,7,8,9,10,10,11],
        'CustomerID': [1,1,2,3,3,1,4,5,5,2,6,3,7,7,4],
        'InvoiceDate': pd.to_datetime(['2024-01-05','2024-01-05','2024-01-12','2024-02-03','2024-02-03','2024-02-20',
                                       '2024-02-25','2024-03-02','2024-03-02','2024-03-15','2024-03-20','2024-04-01',
                                       '2024-04-10','2024-04-10','2024-04-18']),
        'Quantity': [2,3,10,4,1,6,2,5,5,8,3,2,1,4,6],
        'UnitPrice': [3.0,2.0,1.5,5.0,10.0,2.5,12.0,1.0,2.0,3.5,9.0,4.0,20.0,2.5,3.0],
        'Country': ['United Kingdom']*6 + ['France']*3 + ['United Kingdom','Germany','United Kingdom','France','France','Germany'],
    })
    clean['Revenue'] = clean['Quantity'] * clean['UnitPrice']
    order_value = clean.groupby('InvoiceNo')['Revenue'].sum()
    monthly = clean.groupby(clean['InvoiceDate'].dt.to_period('M'))['Revenue'].sum()
    print('Customers:', clean['CustomerID'].nunique(), 'Invoices:', clean['InvoiceNo'].nunique())
    print('Total revenue:', round(clean['Revenue'].sum(), 2))
    print('Average order value:', round(order_value.mean(), 2))
    print('Busiest month:', monthly.idxmax(), round(monthly.max(), 2))
    print(clean.groupby('Country')['Revenue'].sum().sort_values(ascending=False).head(2))
    
  2. From a 20-row clean transactions DataFrame (6 customers, several products), build an RFM table, score Recency and Frequency 1-4 (use rank first so ties are safe), find Champions (R >= 3 and F >= 3), compute each customer's 12-month predicted CLV (AOV x frequency per month x 12), do an ABC tier-A share, and verify the RFM monetary total equals total revenue.

    💡 Hint

    Reuse qcut on ranked values, and a groupby for product revenue before cumulative percentage.

    ✅ Show solution
    import pandas as pd
    
    clean = pd.DataFrame({
        'InvoiceNo': range(1, 21),
        'CustomerID': [1,1,1,2,2,3,3,3,3,4,4,5,5,5,6,6,1,2,3,5],
        'StockCode': ['A','B','A','C','A','D','A','B','E','A','C','B','D','A','F','A','A','B','A','C'],
        'InvoiceDate': pd.to_datetime(['2024-01-03','2024-01-20','2024-02-15','2024-01-10','2024-03-01','2024-01-05','2024-01-25',
                                       '2024-02-10','2024-03-05','2024-02-02','2024-02-28','2024-03-10','2024-03-15','2024-03-20',
                                       '2024-01-08','2024-01-09','2024-03-25','2024-03-28','2024-03-29','2024-03-30']),
        'Revenue': [100,40,120,60,80,30,90,45,20,150,70,25,35,110,15,20,130,50,95,65],
    })
    snapshot = clean['InvoiceDate'].max() + pd.Timedelta(days=1)
    rfm = clean.groupby('CustomerID').agg(
        Recency=('InvoiceDate', lambda s: (snapshot - s.max()).days),
        Frequency=('InvoiceNo', 'nunique'),
        Monetary=('Revenue', 'sum'),
    )
    rfm['R_score'] = pd.qcut(rfm['Recency'].rank(method='first'), 4, labels=[4, 3, 2, 1]).astype(int)
    rfm['F_score'] = pd.qcut(rfm['Frequency'].rank(method='first'), 4, labels=[1, 2, 3, 4]).astype(int)
    champions = rfm[(rfm['R_score'] >= 3) & (rfm['F_score'] >= 3)]
    print('Champions:', list(champions.index), 'worth', champions['Monetary'].sum())
    
    months = (clean['InvoiceDate'].max() - clean['InvoiceDate'].min()).days / 30
    rfm['AOV'] = rfm['Monetary'] / rfm['Frequency']
    rfm['PredictedCLV'] = rfm['AOV'] * (rfm['Frequency'] / months) * 12
    print(rfm['PredictedCLV'].round(1))
    
    product_revenue = clean.groupby('StockCode')['Revenue'].sum().sort_values(ascending=False)
    cum_pct = product_revenue.cumsum() / product_revenue.sum() * 100
    print('Tier A share of products:', round((cum_pct <= 80).sum() / len(product_revenue) * 100, 1), '%')
    print('Reconciles:', abs(rfm['Monetary'].sum() - clean['Revenue'].sum()) < 1)
    
  3. Create 12 weeks of weekly revenue (a gentle upward trend with some noise). Fit a Holt exponential smoothing model with an additive trend, print next week's forecast and the mean of the residuals, then assemble a metrics DataFrame and a short executive summary text with total revenue, last week's revenue, the forecast and a one-line recommendation.

    💡 Hint

    Use statsmodels.tsa.holtwinters.ExponentialSmoothing(weekly, trend='add').fit() and forecast(1).

    ✅ Show solution
    import numpy as np
    import pandas as pd
    from statsmodels.tsa.holtwinters import ExponentialSmoothing
    
    weekly = pd.Series(
        [1000, 1080, 1050, 1150, 1200, 1170, 1280, 1330, 1300, 1400, 1460, 1430],
        index=pd.date_range('2024-01-07', periods=12, freq='W'),
    )
    model = ExponentialSmoothing(weekly, trend='add', seasonal=None).fit()
    next_week = model.forecast(1).iloc[0]
    residuals = weekly - model.fittedvalues
    print('Next week forecast:', round(next_week, 0))
    print('Mean residual:', round(residuals.mean(), 1))
    
    metrics = pd.DataFrame({
        'Metric': ['Total Revenue', 'Last Week Revenue', 'Next Week Forecast'],
        'Value': [weekly.sum(), weekly.iloc[-1], round(next_week, 0)],
    })
    print(metrics)
    
    summary_lines = [
        'EXECUTIVE SUMMARY',
        f'Total revenue over {len(weekly)} weeks: {weekly.sum():,.0f}',
        f'Last week: {weekly.iloc[-1]:,.0f}; forecast for next week: {next_week:,.0f}',
    ]
    if next_week > weekly.iloc[-1]:
        summary_lines.append('Recommendation: revenue is trending up, so keep stock levels ready for growth.')
    else:
        summary_lines.append('Recommendation: revenue is flattening, so review promotions.')
    print('\n'.join(summary_lines))
    

Found this useful?

All lessons, notebooks and datasets here are free. If they helped you, a coffee keeps new lessons coming.