Mathew K Analytics

Lesson 2 · Real-World Data Analytics

RFM Analysis in Python: Customer Segmentation with Pandas | Data Analytics #2

Who are your best customers, and who is about to leave? We score 4,000+ real retail customers on Recency, Frequency and Monetary value with pandas, turn the…

What you'll learn

Datasets used in this lesson

Save these next to the notebook. In Google Colab, upload them with the 📁 icon on the left first.

📓 Full notebook

Download .ipynb

Data Analytics 100, Video 2: Customer Segmentation with RFM Analysis#

  • Video two of the hundred-video real-world data analytics series.
  • Real Recency, Frequency, and Monetary scoring, on the real cleaned retail data from last video.
  • Let's get into it.

This lesson is for learners who have a clean transaction table and now want to turn it into customer insight. The business question is: which customers matter most, which are drifting away, and where should marketing effort go? You will use RFM analysis, a classic method that scores each customer on Recency (how recently they bought), Frequency (how many separate orders they placed) and Monetary value (how much they spent in total). The data is the cleaned UK online gift retailer file saved in the previous video, online_retail_clean.csv, using the CustomerID, InvoiceNo, InvoiceDate, Revenue and Country columns.

In this lesson you will:

  • Build one RFM row per customer with groupby aggregations.
  • Turn raw values into 1 to 4 scores with pd.qcut.
  • Write a rule-based function that assigns named segments.
  • Compare each segment's share of customers with its share of revenue, and save the result.

Prerequisites: you should be comfortable with pandas groupby and filtering, and have the cleaned file from Video 1. Watch that lesson first if not.

Part 1: What RFM Actually Measures#

RFM rests on a simple idea: customers who bought recently, buy often and spend a lot are the most valuable, and each of those three signals can be measured from a transaction log. Everything here starts from the cleaned data, so the code reads online_retail_clean.csv with parse_dates=['InvoiceDate'] to keep dates usable. The output (391150, 9) matches the cleaned row count from the previous video, with the extra Revenue column making nine columns. Checking the shape on reload is a quick way to confirm you are using the right file.

import pandas as pd
clean = pd.read_csv('online_retail_clean.csv', parse_dates=['InvoiceDate'])
clean.shape
(391150, 9)

Part 2: Choosing a Real Snapshot Date#

Recency needs a reference point: 'days since last purchase' as of when? In a historical dataset you cannot use today's date, or every customer would look years inactive. The code takes the latest InvoiceDate in the data and adds one day with pd.Timedelta(days=1). The output is 2011-12-10 12:50, the day after the final transaction. Adding a day means even the most recent buyer gets a recency of at least 1 rather than 0. Tip: always fix the snapshot date once and reuse it, so your scores do not change between runs.

snapshot_date = clean['InvoiceDate'].max() + pd.Timedelta(days=1)
snapshot_date
Timestamp('2011-12-10 12:50:00')

Part 3: Real Recency per Customer#

Recency is often the strongest single predictor of whether a customer will buy again. groupby('CustomerID')['InvoiceDate'].max() finds each customer's last purchase. Subtracting that from snapshot_date gives a time gap, and .dt.days turns it into whole days. describe() summarises the result: 4,334 customers, a median of 51 days and a maximum of 374. The mean, 92.7 days, is well above the median, which tells you a long tail of customers have not bought in many months. Lower recency is better, which matters when you score it later.

recency = clean.groupby('CustomerID')['InvoiceDate'].max()
recency = (snapshot_date - recency).dt.days
recency.describe()
count    4334.000000
mean       92.703046
std       100.177047
min         1.000000
25%        18.000000
50%        51.000000
75%       143.000000
max       374.000000
Name: InvoiceDate, dtype: float64

Part 4: Real Frequency per Customer#

Frequency measures how engaged a customer is. You count distinct orders, not rows, because one invoice can contain many product lines. groupby('CustomerID')['InvoiceNo'].nunique() does exactly that. The summary shows the median customer placed 2 orders and at least a quarter placed only 1, while the maximum is 206. The mean of about 4.2 sits above the median because a few heavy buyers pull it up. Common mistake: using .count() or .size() here would count product lines and badly overstate frequency.

frequency = clean.groupby('CustomerID')['InvoiceNo'].nunique()
frequency.describe()
count    4334.000000
mean        4.245962
std         7.634989
min         1.000000
25%         1.000000
50%         2.000000
75%         5.000000
max       206.000000
Name: InvoiceNo, dtype: float64

Part 5: Real Monetary Value per Customer#

Monetary value captures how much each customer is worth in revenue. groupby('CustomerID')['Revenue'].sum() adds up every line each customer bought. The summary is very skewed: the median customer spent 662.57, but the mean is 2,015.97 and the top customer spent 279,138.02. The standard deviation, 8,903.67, is far larger than the mean, another sign of a few very large accounts. This skew is why the scoring step uses quartiles rather than fixed spending bands.

monetary = clean.groupby('CustomerID')['Revenue'].sum()
monetary.describe()
count      4334.000000
mean       2015.973152
std        8903.673825
min           3.750000
25%         304.240000
50%         662.565000
75%        1631.622500
max      279138.020000
Name: Revenue, dtype: float64

Part 6: Combining Into One Real RFM Table#

Having three separate Series is awkward, so you combine them into one table with one row per customer. pd.DataFrame({...}) lines the three up automatically by their shared CustomerID index. reset_index() then turns CustomerID back into a normal column, which makes saving and merging easier. rfm.head() is not displayed because it is not the last line; the output is rfm.shape[0], which is 4,334 customers, matching the count from each summary.

rfm = pd.DataFrame({'Recency': recency, 'Frequency': frequency, 'Monetary': monetary})
rfm = rfm.reset_index()
rfm.head()
rfm.shape[0]
4334

Part 7: Confirming the Real Customer Count Lines Up#

Before scoring, you confirm nothing was lost or misaligned when combining the three measures. The first line checks that the RFM table has as many rows as there are unique customers in the cleaned data. That line is not displayed, so run it on its own to see True. The displayed line, rfm.isna().sum().sum(), totals the missing values across all columns and returns 0. Zero blanks means every customer has a recency, a frequency and a monetary value, so the three Series lined up perfectly.

rfm.shape[0] == clean['CustomerID'].nunique()
rfm.isna().sum().sum()
np.int64(0)

Part 8: Scoring Recency Into Real Quartiles#

Raw recency in days is hard to compare with spend in pounds, so you convert each measure into a simple 1 to 4 score. pd.qcut(..., 4) splits customers into four groups of roughly equal size, called quartiles. Because a low recency is good, the labels are reversed, [4, 3, 2, 1], so the most recent quarter scores 4. .astype(int) turns the category labels into numbers you can add. The output shows four groups of 1,058 to 1,124 customers; they are not exactly equal because many customers share the same recency value.

rfm['R_score'] = pd.qcut(rfm['Recency'], 4, labels=[4, 3, 2, 1]).astype(int)
rfm['R_score'].value_counts().sort_index()
R_score
1    1079
2    1073
3    1058
4    1124
Name: count, dtype: int64

Part 9: Scoring Frequency Into Real Quartiles#

Frequency needs a trick. So many customers have exactly 1 or 2 orders that quartile edges would repeat, and pd.qcut would raise an error about duplicate bin edges. The code first applies .rank(method='first'), which gives every customer a unique rank, breaking ties by the order they appear. qcut then splits those ranks into four equal groups, labelled 1 to 4 with higher meaning more frequent. The output shows almost perfectly even groups of 1,083 or 1,084. The trade-off: two customers with the same frequency can land in different score groups.

rfm['F_score'] = pd.qcut(rfm['Frequency'].rank(method='first'), 4, labels=[1, 2, 3, 4]).astype(int)
rfm['F_score'].value_counts().sort_index()
F_score
1    1084
2    1083
3    1083
4    1084
Name: count, dtype: int64

Part 10: Scoring Monetary Into Real Quartiles#

Monetary values are continuous and rarely tied, so pd.qcut works directly without ranking. The code splits customers into four spending quartiles labelled 1 (lowest) to 4 (highest) and converts the labels to integers. The output shows groups of 1,082 to 1,085 customers, close to a quarter of 4,334 each. Each M score now means 'this customer is in the top, second, third or bottom quarter of spenders', which is easier to act on than a raw pound figure.

rfm['M_score'] = pd.qcut(rfm['Monetary'], 4, labels=[1, 2, 3, 4]).astype(int)
rfm['M_score'].value_counts().sort_index()
M_score
1    1085
2    1082
3    1083
4    1084
Name: count, dtype: int64

Part 11: A Real Combined RFM Score#

A quick way to rank customers overall is to add the three scores together. RFM_Sum ranges from 3 (scored 1 on everything) to 12 (scored 4 on everything). describe() shows a mean of about 7.5 and a median of 7, with the middle half of customers between 5 and 10. The weakness of a plain sum is that very different customers can tie: a recent one-time buyer and a big spender who has gone quiet might both score 8. That is why the next step builds named segments from the individual scores.

rfm['RFM_Sum'] = rfm['R_score'] + rfm['F_score'] + rfm['M_score']
rfm['RFM_Sum'].describe()
count    4334.000000
mean        7.513613
std         2.828945
min         3.000000
25%         5.000000
50%         7.000000
75%        10.000000
max        12.000000
Name: RFM_Sum, dtype: float64

Part 12: A Real Rule-Based Segment Function#

Named segments are easier for a marketing team to use than numeric scores. The function looks at one customer row and checks rules in order, returning the first match: high on all three scores is Champions; recent with at least moderate frequency is Loyal Customers; recent but otherwise low is New Customers; an R score of 2 is At Risk; everything else is Lost. The order of the if and elif checks matters, because a customer stops at the first rule that fits. Defining the function produces no output.

def segment_customer(row):
    if row['R_score'] >= 3 and row['F_score'] >= 3 and row['M_score'] >= 3:
        return 'Champions'
    elif row['R_score'] >= 3 and row['F_score'] >= 2:
        return 'Loyal Customers'
    elif row['R_score'] >= 3:
        return 'New Customers'
    elif row['R_score'] == 2:
        return 'At Risk'
    else:
        return 'Lost'

Part 13: Applying the Real Segmentation#

Now each customer gets a label. rfm.apply(segment_customer, axis=1) runs the function once per row; axis=1 means 'pass each row', not each column. value_counts() then counts customers per segment. Champions is the biggest group with 1,315 customers, followed by Lost (1,079) and At Risk (1,073). Loyal Customers has 607 and New Customers 260. Notice Lost exactly matches the 1,079 customers with an R score of 1 and At Risk matches the 1,073 with R score 2, which confirms the rules worked as intended.

rfm['Segment'] = rfm.apply(segment_customer, axis=1)
rfm['Segment'].value_counts()
Segment
Champions          1315
Lost               1079
At Risk            1073
Loyal Customers     607
New Customers       260
Name: count, dtype: int64

Part 14: Real Revenue Contribution by Segment#

Segment sizes only tell half the story; the business cares where the money comes from. The code sums Monetary by segment and sorts it. Only the last line is displayed: each segment's total divided by the overall total, times 100. Champions bring in 73.0% of revenue. At Risk is next with 12.3%, ahead of Lost at 8.0% and Loyal Customers at 5.7%, while New Customers contribute just 1.0%. This is the classic pattern where a minority of customers drives most of the revenue.

segment_revenue = rfm.groupby('Segment')['Monetary'].sum().sort_values(ascending=False)
segment_revenue
(segment_revenue / segment_revenue.sum() * 100).round(1)
Segment
Champions          73.0
At Risk            12.3
Lost                8.0
Loyal Customers     5.7
New Customers       1.0
Name: Monetary, dtype: float64

Part 15: Real Average Behavior by Segment#

Averages per segment let you describe each group in plain words for a stakeholder. The code groups by Segment and takes the mean() of all three raw measures. Champions bought on average 17.0 days ago, placed 9.3 orders and spent 4,847.8. Lost customers last bought about 247.8 days ago and placed just 1.6 orders. At Risk customers are interesting: an average spend of 1,004.9 is the second highest, but they have not ordered for about 85 days. Remember that means are sensitive to a few very large accounts.

rfm.groupby('Segment')[['Recency', 'Frequency', 'Monetary']].mean().round(1)
Recency Frequency Monetary
Segment
At Risk 84.6 2.6 1004.9
Champions 17.0 9.3 4847.8
Lost 247.8 1.6 645.5
Loyal Customers 23.3 2.2 820.0
New Customers 27.1 1.0 345.7

Part 16: Real Customers vs Real Revenue Share#

Putting customer share next to revenue share shows which segments punch above their weight. The code converts both counts and revenue into percentages and places them side by side in a DataFrame; pandas aligns the rows by segment name. Champions are 30.3% of customers but 73.0% of revenue. Lost customers are 24.9% of customers but only 8.0% of revenue. At Risk is 24.8% of customers and 12.3% of revenue. When revenue share is much higher than customer share, losing even a few of those customers hurts.

segment_customers = rfm['Segment'].value_counts()
customer_share = (segment_customers / segment_customers.sum() * 100).round(1)
revenue_share = (segment_revenue / segment_revenue.sum() * 100).round(1)
pd.DataFrame({'Customer %': customer_share, 'Revenue %': revenue_share})
Customer % Revenue %
Segment
At Risk 24.8 12.3
Champions 30.3 73.0
Lost 24.9 8.0
Loyal Customers 14.0 5.7
New Customers 6.0 1.0

Part 17: Visualizing Real Segment Sizes#

A bar chart makes segment sizes easy to present. The code stores the segment order from value_counts().index, so the bars run from largest to smallest, and uses that same order to pick the heights. plt.bar draws the chart, which is then saved as rfm_segment_sizes.png and closed, so nothing appears in the notebook. Open the image and look for Champions as the tallest bar, Lost and At Risk at similar heights, and New Customers as the smallest. Sorting bars by size almost always makes a chart easier to read.

import matplotlib.pyplot as plt
order = rfm['Segment'].value_counts().index
plt.figure(figsize=(8, 5))
plt.bar(order, rfm['Segment'].value_counts()[order], color='seagreen')
plt.title('Real Customer Count by RFM Segment')
plt.ylabel('Number of Real Customers')
plt.tight_layout()
plt.savefig('rfm_segment_sizes.png', dpi=120)
plt.close()

Part 18: Visualizing Real Recency Against Monetary Value#

A scatter plot shows how individual customers spread across segments. The code loops over rfm.groupby('Segment') and draws each segment in its own colour from the colors dictionary, with alpha=0.6 to make overlapping dots see-through. plt.yscale('log') puts monetary value on a log scale, where each step up is a multiplication, so small and huge spenders fit on one chart. The figure is saved to rfm_recency_monetary_scatter.png. Look for gold Champions clustered at low recency, grey Lost customers on the far right, and clear vertical bands where the recency quartile edges sit.

colors = {'Champions': 'gold', 'Loyal Customers': 'seagreen', 'New Customers': 'steelblue', 'At Risk': 'orange', 'Lost': 'gray'}
plt.figure(figsize=(9, 6))
for seg, group in rfm.groupby('Segment'):
    plt.scatter(group['Recency'], group['Monetary'], label=seg, alpha=0.6, color=colors[seg])
plt.yscale('log')
plt.xlabel('Real Recency (days since last order)')
plt.ylabel('Real Monetary Value (log scale)')
plt.title('Real Recency vs Monetary Value by Segment')
plt.legend()
plt.tight_layout()
plt.savefig('rfm_recency_monetary_scatter.png', dpi=120)
plt.close()

Part 19: Real Champions Worth Naming#

A list of named top customers is what an account manager actually acts on. The code filters to Champions, sorts by Monetary from highest to lowest and shows the top ten. Customer 14646 leads with 279,138.02 from 72 orders, and customer 14911 stands out with 198 orders. Most of these customers score 4, 4, 4 for a combined 12, though 12415 and 16029 score 11 because their recency is a little older. These accounts deserve personal attention, such as early access or a dedicated contact.

rfm[rfm['Segment'] == 'Champions'].sort_values('Monetary', ascending=False).head(10)
CustomerID Recency Frequency Monetary R_score F_score M_score RFM_Sum Segment
1689 14646 2 72 279138.02 4 4 4 12 Champions
4197 18102 1 60 259657.30 4 4 4 12 Champions
3725 17450 8 46 194390.79 4 4 4 12 Champions
1879 14911 1 198 136161.83 4 4 4 12 Champions
55 12415 24 20 124564.53 3 4 4 11 Champions
1333 14156 10 54 116560.08 4 4 4 12 Champions
3768 17511 3 31 91062.38 4 4 4 12 Champions
2700 16029 39 62 72708.09 3 4 4 11 Champions
3174 16684 4 28 66653.56 4 4 4 12 Champions
996 13694 4 50 65039.62 4 4 4 12 Champions

Part 20: Real At-Risk Customers Worth a Win-Back#

At Risk customers used to be valuable but have not bought for a while, so they are the best targets for a win-back campaign. The code filters to At Risk and sorts by spend. Customer 12939 tops the list with 11,581.80 across 8 orders but last bought 64 days before the snapshot. Every customer shown has an M score of 4, so they were top-quarter spenders, yet all have an R score of 2. Tip: prioritise win-back offers by value, as this sort does, rather than contacting everyone in the segment equally.

rfm[rfm['Segment'] == 'At Risk'].sort_values('Monetary', ascending=False).head(10)
CustomerID Recency Frequency Monetary R_score F_score M_score RFM_Sum Segment
459 12939 64 8 11581.80 2 4 4 10 At Risk
50 12409 79 3 11072.67 2 3 4 9 At Risk
2812 16180 100 8 10254.18 2 4 4 10 At Risk
324 12744 56 4 9120.39 2 3 4 9 At Risk
1903 14952 60 11 8099.49 2 4 4 10 At Risk
73 12435 80 2 7829.89 2 2 4 8 At Risk
3219 16745 87 17 7180.70 2 4 4 10 At Risk
519 13027 114 6 6912.00 2 4 4 10 At Risk
3150 16652 59 7 6773.97 2 4 4 10 At Risk
2814 16182 72 4 6617.65 2 3 4 9 At Risk

Part 21: Saving the Real RFM Table#

Saving the RFM table lets later lessons, dashboards or a CRM import reuse the scores without recalculating them. to_csv(..., index=False) writes the file without an extra index column. The code reloads it with pd.read_csv and compares shapes. The output True confirms the file has the same rows and columns as the table in memory. Note that the Segment column is saved as plain text, so it reloads cleanly without any special handling.

rfm.to_csv('online_retail_rfm.csv', index=False)
reloaded_rfm = pd.read_csv('online_retail_rfm.csv')
reloaded_rfm.shape == rfm.shape
True

Part 22: One Last Real Sanity Check#

A final check makes sure every customer received exactly one segment. The first line counts missing segment labels (not displayed; run it alone to see 0). The displayed line groups by Segment, counts each group with .size(), adds the counts together and compares with the total number of rows. The output is True: the segment counts add up to all 4,334 customers. If the rule function ever returned nothing for some combination of scores, this check would fail.

rfm['Segment'].isna().sum()
rfm.groupby('Segment').size().sum() == rfm.shape[0]
np.True_

Part 23: Real Correlation Between Recency, Frequency, and Monetary#

Correlation tells you whether the three RFM measures carry different information or repeat each other. .corr() calculates the Pearson correlation, a number from -1 to 1 showing how strongly two columns move together in a straight line. Frequency and Monetary have the strongest link at 0.55: customers who order more tend to spend more. Recency is negatively linked with both, -0.26 with Frequency and -0.12 with Monetary, meaning recent buyers tend to be more frequent. None of the values is close to 1, so all three measures add something useful.

rfm[['Recency', 'Frequency', 'Monetary']].corr().round(2)
Recency Frequency Monetary
Recency 1.00 -0.26 -0.12
Frequency -0.26 1.00 0.55
Monetary -0.12 0.55 1.00

Part 24: Does a Higher Real RFM Score Actually Mean More Spend?#

If the combined score is meaningful, average spend should rise with it. The code groups customers by RFM_Sum and takes the mean of Monetary. The relationship is clear and steady: customers with a score of 3 spent 162.4 on average, those at 7 spent 722.6, and those at the top score of 12 spent 8,826.6. The jump from 11 (3,768.2) to 12 is especially large. This gives you confidence that the scoring captures real value, even though the named segments are more useful for action.

rfm.groupby('RFM_Sum')['Monetary'].mean().round(1)
RFM_Sum
3      162.4
4      251.6
5      367.8
6      673.1
7      722.6
8     1105.4
9     1363.4
10    2279.3
11    3768.2
12    8826.6
Name: Monetary, dtype: float64

Part 25: Real Distribution of the Combined Score#

A histogram shows how many customers fall at each combined score. plt.hist uses bins=range(3, 14), which creates one bin per whole score from 3 to 12, and edgecolor='white' separates the bars. The chart is saved to rfm_score_distribution.png rather than shown. When you open it, look at whether customers are spread evenly or bunched at the low and high ends. With a median of 7 and quartiles at 5 and 10, you should expect a fairly wide spread rather than one sharp peak.

plt.figure(figsize=(8, 5))
plt.hist(rfm['RFM_Sum'], bins=range(3, 14), color='slateblue', edgecolor='white')
plt.xlabel('Real Combined RFM Score')
plt.ylabel('Number of Real Customers')
plt.title('Real Distribution of Combined RFM Scores')
plt.tight_layout()
plt.savefig('rfm_score_distribution.png', dpi=120)
plt.close()

Part 26: Where the Real Champions Actually Are#

Knowing where your best customers are helps with shipping, language and campaign choices. The RFM table has no country, so the code takes each customer's first recorded country with groupby('CustomerID')['Country'].first() and adds it with merge(..., on='CustomerID'). It then counts Champions by country. The United Kingdom dominates with 1,190 of the 1,315 Champions, followed by France (35) and Germany (33). This mirrors the UK-heavy dataset, so any per-country comparison should allow for the much smaller customer numbers abroad.

customer_country = clean.groupby('CustomerID')['Country'].first()
rfm_with_country = rfm.merge(customer_country, on='CustomerID')
rfm_with_country[rfm_with_country['Segment'] == 'Champions']['Country'].value_counts().head(5)
Country
United Kingdom    1190
France              35
Germany             33
Belgium             10
Spain                7
Name: count, dtype: int64

Part 27: The Real Quartile Boundaries Behind Each Score#

Scores are easier to explain when you can say exactly what they mean in days. Passing retbins=True to pd.qcut returns the bin edges as well as the labels; the _ throws away the labels you do not need. The output shows the recency edges: 1, 18, 51, 143 and 374 days. So an R score of 4 means a last purchase within about 18 days of the snapshot, and an R score of 1 means more than 143 days ago. These edges are the same as the quartiles in the earlier recency summary.

_, r_bins = pd.qcut(rfm['Recency'], 4, labels=[4, 3, 2, 1], retbins=True)
r_bins.round(1)
array([  1.,  18.,  51., 143., 374.])

Part 28: Real Median vs Real Mean Spend by Segment#

Comparing median and mean reveals how skewed spending is inside each segment. The code computes both, but only the last line, the mean, is displayed. That output matches the earlier averages: Champions 4,847.8, At Risk 1,004.9, Loyal Customers 820.0, Lost 645.5 and New Customers 345.7. To complete the comparison, run the .median() line on its own. If a segment's median is much lower than its mean, a few very large accounts are pulling the average up, and the median is the fairer 'typical customer' figure.

rfm.groupby('Segment')['Monetary'].median().round(1)
rfm.groupby('Segment')['Monetary'].mean().round(1)
Segment
At Risk            1004.9
Champions          4847.8
Lost                645.5
Loyal Customers     820.0
New Customers       345.7
Name: Monetary, dtype: float64

Part 29: A Real Edge Case Worth Knowing#

Rules can produce surprises, so it is worth checking odd combinations. Here the question is whether any Champion placed only one order, which would sound wrong for a 'best customer'. The code filters for Segment == 'Champions' and Frequency == 1 together, using & with each condition in brackets. The output is 0. Because Champions need an F score of at least 3, and the ranking pushed one-order customers into the lower frequency scores, none slipped through. Tip: in pandas you must use &, not and, to combine conditions.

one_order_champions = rfm[(rfm['Segment'] == 'Champions') & (rfm['Frequency'] == 1)]
one_order_champions.shape[0]
0

Part 30: Reconciling Row Counts One Last Time#

Merges can quietly add or drop rows, for example if a key is duplicated or missing. This check compares the number of rows in rfm_with_country with the original RFM table. The output is True, so the merge with countries kept every customer exactly once. Because customer_country was built with .first(), there is only one country per customer, which is what prevents duplication. Make this check a habit after every merge you do.

rfm_with_country.shape[0] == rfm.shape[0]
True

Wrap-Up: What You Learned#

  • RFM genuinely scores every real customer on Recency, Frequency, and Monetary value, each split into real quartiles.
  • Combining the three real individual scores into rule-based segments beats using just the real combined total alone.
  • In this real dataset, a small real Champions segment genuinely drives a hugely disproportionate share of real revenue.
  • The real At Risk segment is the real highest-value group still worth an active win-back effort.
  • Next video: real market basket analysis, finding which real products actually get bought together.

Practice on your own

  • Run rfm.groupby('Segment')['Monetary'].median() on its own and put it next to the means. Which segment has the biggest gap between median and mean?
  • Print the frequency and monetary quartile edges with retbins=True, just as you did for recency, and write one sentence explaining what an F score of 4 and an M score of 4 mean.
  • Add a rule to segment_customer that labels customers with R score 1 but M score 4 as Cannot Lose Them, reapply it and see how many customers move out of Lost.

The next video in the series covers market basket analysis, finding which products actually get bought together on the same invoice.

🧠 Test yourself

Pick an answer to check it. Your best score is saved in this browser.

  1. In RFM scoring, why is Recency scored with labels=[4, 3, 2, 1] while Frequency uses [1, 2, 3, 4]?

  2. Why does the lesson set snapshot_date = clean['InvoiceDate'].max() + pd.Timedelta(days=1)?

  3. Why is rank(method='first') applied to Frequency before pd.qcut(..., 4)?

  4. Given the lesson's segment_customer rules, what segment does a customer with R_score=4, F_score=3, M_score=1 get?

    if R >= 3 and F >= 3 and M >= 3: 'Champions'
    elif R >= 3 and F >= 2: 'Loyal Customers'
    elif R >= 3: 'New Customers'
    elif R == 2: 'At Risk'
    else: 'Lost'
    
  5. Why does the lesson prefer rule-based segments over using only the combined RFM_Sum score?

💻 Practice exercises

Try each one in the notebook or in your own editor before opening the solution.

  1. Create a 15-row transactions DataFrame (InvoiceNo, CustomerID, InvoiceDate, Revenue) for 5 customers. Set snapshot_date as the max date plus one day and build an rfm table with Recency (days), Frequency (distinct invoices) and Monetary (sum of Revenue) per customer.

    💡 Hint

    Use groupby('CustomerID').agg(...) with a lambda for recency.

    ✅ Show solution
    import pandas as pd
    
    tx = pd.DataFrame({
        'InvoiceNo': range(1, 16),
        'CustomerID': [1,1,1,2,2,3,3,3,3,4,4,5,5,5,5],
        'InvoiceDate': pd.to_datetime(['2024-01-05','2024-02-10','2024-03-20','2024-01-15','2024-01-28',
                                       '2024-02-01','2024-02-15','2024-03-01','2024-03-25','2024-01-02',
                                       '2024-01-20','2024-03-10','2024-03-18','2024-03-22','2024-03-28']),
        'Revenue': [50,70,30,200,150,20,25,30,40,300,100,15,20,25,30],
    })
    snapshot_date = tx['InvoiceDate'].max() + pd.Timedelta(days=1)
    rfm = tx.groupby('CustomerID').agg(
        Recency=('InvoiceDate', lambda s: (snapshot_date - s.max()).days),
        Frequency=('InvoiceNo', 'nunique'),
        Monetary=('Revenue', 'sum'),
    ).reset_index()
    print(rfm)
    
  2. Using an RFM table of 12 customers (Recency, Frequency, Monetary typed in by hand), score each metric 1-4 with pd.qcut (Recency reversed, Frequency ranked first), add RFM_Sum, then use a segment_customer function with the lesson's rules to assign segments. Print the segment counts.

    💡 Hint

    Score Recency with labels [4,3,2,1] and use rank(method='first') for Frequency.

    ✅ Show solution
    import pandas as pd
    
    rfm = pd.DataFrame({
        'CustomerID': range(1, 13),
        'Recency':   [5, 12, 30, 45, 60, 90, 120, 150, 200, 250, 300, 340],
        'Frequency': [12, 8, 6, 5, 4, 3, 2, 2, 1, 1, 1, 1],
        'Monetary':  [2500, 1800, 900, 1200, 600, 450, 300, 250, 120, 90, 60, 40],
    })
    rfm['R_score'] = pd.qcut(rfm['Recency'].rank(method='first'), 4, labels=[4, 3, 2, 1]).astype(int)
    rfm['F_score'] = pd.qcut(rfm['Frequency'].rank(method='first'), 4, labels=[1, 2, 3, 4]).astype(int)
    rfm['M_score'] = pd.qcut(rfm['Monetary'].rank(method='first'), 4, labels=[1, 2, 3, 4]).astype(int)
    rfm['RFM_Sum'] = rfm['R_score'] + rfm['F_score'] + rfm['M_score']
    
    def segment_customer(row):
        if row['R_score'] >= 3 and row['F_score'] >= 3 and row['M_score'] >= 3:
            return 'Champions'
        elif row['R_score'] >= 3 and row['F_score'] >= 2:
            return 'Loyal Customers'
        elif row['R_score'] >= 3:
            return 'New Customers'
        elif row['R_score'] == 2:
            return 'At Risk'
        else:
            return 'Lost'
    
    rfm['Segment'] = rfm.apply(segment_customer, axis=1)
    print(rfm['Segment'].value_counts())
    
  3. Reuse a hand-made scored RFM table (12 customers with Segment and Monetary already assigned in the code). Build a comparison table with each segment's Customer % and Revenue %, plus average Recency, Frequency and Monetary per segment. Print which segment has the biggest gap between revenue share and customer share.

    💡 Hint

    Divide value_counts() and grouped sums by their totals, then combine into one DataFrame.

    ✅ Show solution
    import pandas as pd
    
    rfm = pd.DataFrame({
        'CustomerID': range(1, 13),
        'Recency':   [5, 12, 30, 45, 60, 90, 120, 150, 200, 250, 300, 340],
        'Frequency': [12, 8, 6, 5, 4, 3, 2, 2, 1, 1, 1, 1],
        'Monetary':  [2500, 1800, 900, 1200, 600, 450, 300, 250, 120, 90, 60, 40],
        'Segment':   ['Champions','Champions','Champions','Loyal Customers','Loyal Customers','At Risk',
                      'At Risk','At Risk','Lost','Lost','Lost','Lost'],
    })
    customer_share = (rfm['Segment'].value_counts() / len(rfm) * 100).round(1)
    revenue = rfm.groupby('Segment')['Monetary'].sum()
    revenue_share = (revenue / revenue.sum() * 100).round(1)
    compare = pd.DataFrame({'Customer %': customer_share, 'Revenue %': revenue_share})
    compare['Gap'] = compare['Revenue %'] - compare['Customer %']
    print(compare)
    print(rfm.groupby('Segment')[['Recency', 'Frequency', 'Monetary']].mean().round(1))
    print('Biggest gap:', compare['Gap'].idxmax())
    

Found this useful?

All lessons, notebooks and datasets here are free. If they helped you, a coffee keeps new lessons coming.