Mathew K Analytics

Lesson 29 · Market Research Analytics in Python

Rule Based Customer Segmentation in Python: Step-by-Step Training

We are learning to segment customers using rules based on real business data. Segmentation helps companies target the right customers and improve products…

⬇ Download notebookOpen in Colab ↗

What you'll learn

Data

No separate download needed — the notebook creates or downloads everything it uses.

📓 Full notebook

Download .ipynb

Rule-Based Customer Segmentation in Market Research#

  • We are learning to segment customers using rules based on real business data.
  • Segmentation helps companies target the right customers and improve products or marketing.
  • You will use real survey and behavioral data to design and analyze practical customer segments.
  • By the end, you will have learned to define rules, apply them to data, and interpret actionable insights.
import pandas as pd
import numpy as np
import openml
import warnings
warnings.filterwarnings('ignore')

Understanding the Data Used for Rule-Based Segmentation#

  • Market research data includes customer demographics, surveys, and behavior.
  • Each row represents a person or customer with features like age, gender, and service usage.
  • Beginners often forget to handle missing values or misinterpret text responses.
  • Columns may be categorical, numerical, or text, each requiring different analysis techniques.
  • Getting to know your data structure is essential before creating any rules.
dataset = openml.datasets.get_dataset(42178)
df_satisfaction, _, _, _ = dataset.get_data(dataset_format='dataframe')
print(df_satisfaction.shape)
print(df_satisfaction.head(3))
(7043, 20)
   gender  SeniorCitizen Partner Dependents  tenure PhoneService  \
0  Female              0     Yes         No       1           No   
1    Male              0      No         No      34          Yes   
2    Male              0      No         No       2          Yes   

      MultipleLines InternetService OnlineSecurity OnlineBackup  \
0  No phone service             DSL             No          Yes   
1                No             DSL            Yes           No   
2                No             DSL            Yes          Yes   

  DeviceProtection TechSupport StreamingTV StreamingMovies        Contract  \
0               No          No          No              No  Month-to-month   
1              Yes          No          No              No        One year   
2               No          No          No              No  Month-to-month   

  PaperlessBilling     PaymentMethod  MonthlyCharges TotalCharges Churn  
0              Yes  Electronic check           29.85        29.85    No  
1               No      Mailed check           56.95       1889.5    No  
2              Yes      Mailed check           53.85       108.15   Yes  
churn_counts = df_satisfaction['Churn'].value_counts()
print('Churned vs Not Churned:')
print(churn_counts)
Churned vs Not Churned:
Churn
No     5174
Yes    1869
Name: count, dtype: int64
contract_counts = df_satisfaction['Contract'].value_counts()
print('Customers by Contract Type:')
print(contract_counts)
Customers by Contract Type:
Contract
Month-to-month    3875
Two year          1695
One year          1473
Name: count, dtype: int64
df_satisfaction['Segment'] = np.where(df_satisfaction['tenure'] < 12, 'New', 'Established')
print(df_satisfaction[['tenure','Segment']].head(5))
   tenure      Segment
0       1          New
1      34  Established
2       2          New
3      45  Established
4       2          New
churn_by_segment = pd.crosstab(df_satisfaction['Segment'], df_satisfaction['Churn'], normalize='index')
print(churn_by_segment)
Churn              No       Yes
Segment                        
Established  0.825090  0.174910
New          0.517158  0.482842
url = 'https://archive.ics.uci.edu/ml/machine-learning-databases/00502/online_retail_II.xlsx'
df_retail = pd.read_excel(url, sheet_name='Year 2010-2011')
df_retail['InvoiceDate'] = pd.to_datetime(df_retail['InvoiceDate'])
print(df_retail.shape)
print(df_retail[['Customer ID','InvoiceDate','Country']].head(3))
(541910, 8)
   Customer ID         InvoiceDate         Country
0      17850.0 2010-12-01 08:26:00  United Kingdom
1      17850.0 2010-12-01 08:26:00  United Kingdom
2      17850.0 2010-12-01 08:26:00  United Kingdom
customers_by_country = df_retail.groupby('Country')['Customer ID'].nunique().sort_values(ascending=False)
print(customers_by_country.head(5))
Country
United Kingdom    3950
Germany             95
France              87
Spain               31
Belgium             25
Name: Customer ID, dtype: int64
customer_value = df_retail.groupby('Customer ID')['Quantity'].sum()
value_threshold = customer_value.quantile(0.9)
high_value_ids = customer_value[customer_value > value_threshold].index
df_retail['ValueSegment'] = np.where(df_retail['Customer ID'].isin(high_value_ids), 'High Value', 'Regular')
print(df_retail[['Customer ID','Quantity','ValueSegment']].drop_duplicates().head(7))
    Customer ID  Quantity ValueSegment
0       17850.0         6      Regular
2       17850.0         8      Regular
5       17850.0         2      Regular
9       13047.0         6      Regular
10      13047.0         3      Regular
13      13047.0        32      Regular
16      13047.0         8      Regular
df_satisfaction['Segment2'] = np.select(
    [
        (df_satisfaction['tenure'] < 12) & (df_satisfaction['Contract']=='Month-to-month'),
        (df_satisfaction['tenure'] >= 12) & (df_satisfaction['Contract']=='Two year')
    ],
    ['At-Risk New', 'Loyal Long-Term'],
    default='Other'
)
print(df_satisfaction[['tenure','Contract','Segment2']].head(7))
   tenure        Contract     Segment2
0       1  Month-to-month  At-Risk New
1      34        One year        Other
2       2  Month-to-month  At-Risk New
3      45        One year        Other
4       2  Month-to-month  At-Risk New
5       8  Month-to-month  At-Risk New
6      22  Month-to-month        Other
multi_seg_churn = pd.crosstab(df_satisfaction['Segment2'], df_satisfaction['Churn'], normalize='index')
print(multi_seg_churn)
Churn                  No       Yes
Segment2                           
At-Risk New      0.480608  0.519392
Loyal Long-Term  0.970660  0.029340
Other            0.762789  0.237211
np.random.seed(42)
df_nps = pd.DataFrame({
    'CustomerID': range(1,501),
    'Age': np.random.randint(18,70,500),
    'Region': np.random.choice(['North','South','East','West'],500),
    'NPS_Score': np.random.randint(0,11,500)
})
print(df_nps.shape)
print(df_nps.head(3))
(500, 4)
   CustomerID  Age Region  NPS_Score
0           1   56   West          2
1           2   69  North          0
2           3   46   East          4
def nps_segment(score):
    if score <= 6:
        return 'Detractor'
    elif score <= 8:
        return 'Passive'
    else:
        return 'Promoter'
df_nps['NPS_Segment'] = df_nps['NPS_Score'].apply(nps_segment)
print(df_nps.groupby('NPS_Segment').size())
NPS_Segment
Detractor    323
Passive       92
Promoter      85
dtype: int64
nps_by_region = pd.crosstab(df_nps['Region'], df_nps['NPS_Segment'], normalize='index')
print(nps_by_region.round(2))
NPS_Segment  Detractor  Passive  Promoter
Region                                   
East              0.67     0.18      0.15
North             0.58     0.26      0.17
South             0.69     0.16      0.15
West              0.66     0.13      0.21
summary = df_nps.groupby(['Region','NPS_Segment']).size().unstack().fillna(0)
summary.to_csv('nps_segment_summary.csv')
print('Segmentation summary file saved as nps_segment_summary.csv')
Segmentation summary file saved as nps_segment_summary.csv
missing = df_retail.isnull().sum()
print('Missing values per column:')
print(missing[missing > 0])
Missing values per column:
Description      1454
Customer ID    135080
dtype: int64
try:
    result = df_retail.groupby('ValuSegment')['Customer ID'].nunique()
except KeyError as e:
    print('Error:', e)
    print('Check for correct segment column name: should be ValueSegment')
Error: 'ValuSegment'
Check for correct segment column name: should be ValueSegment
def safe_nps_segment(score):
    if pd.isnull(score) or not (0 <= score <= 10):
        return 'Unknown'
    elif score <= 6:
        return 'Detractor'
    elif score <= 8:
        return 'Passive'
    else:
        return 'Promoter'
df_nps['Safe_NPS_Segment'] = df_nps['NPS_Score'].apply(safe_nps_segment)
print(df_nps['Safe_NPS_Segment'].value_counts())
Safe_NPS_Segment
Detractor    323
Passive       92
Promoter      85
Name: count, dtype: int64
tenure_contract_ct = pd.crosstab(df_satisfaction['Segment'], df_satisfaction['Contract'])
print(tenure_contract_ct)
Contract     Month-to-month  One year  Two year
Segment                                        
Established            1967      1371      1636
New                    1908       102        59
df_satisfaction['LoyaltyIndex'] = df_satisfaction['tenure'] * (df_satisfaction['Contract']=='Two year').astype(int)
print(df_satisfaction[['tenure','Contract','LoyaltyIndex']].head(6))
   tenure        Contract  LoyaltyIndex
0       1  Month-to-month             0
1      34        One year             0
2       2  Month-to-month             0
3      45        One year             0
4       2  Month-to-month             0
5       8  Month-to-month             0
df_retail['Month'] = df_retail['InvoiceDate'].dt.to_period('M')
monthly_counts = df_retail.groupby('Month')['Customer ID'].nunique()
print(monthly_counts.tail(12))
Month
2011-01     783
2011-02     798
2011-03    1020
2011-04     899
2011-05    1079
2011-06    1051
2011-07     993
2011-08     980
2011-09    1302
2011-10    1425
2011-11    1711
2011-12     686
Freq: M, Name: Customer ID, dtype: int64
segment_counts = df_satisfaction['Segment2'].value_counts()
at_risk_churn = multi_seg_churn.loc['At-Risk New', 'Yes'] if 'At-Risk New' in multi_seg_churn.index else None
if at_risk_churn is not None and at_risk_churn > 0.3:
    recommendation = 'Launch onboarding and specials for At-Risk New to reduce churn.'
else:
    recommendation = 'Focus on other segments or maintain current actions.'
print('Segment sizes:')
print(segment_counts)
print('At-Risk New churn rate:', at_risk_churn)
print('Recommendation:', recommendation)
Segment sizes:
Segment2
Other              3499
At-Risk New        1908
Loyal Long-Term    1636
Name: count, dtype: int64
At-Risk New churn rate: 0.5193920335429769
Recommendation: Launch onboarding and specials for At-Risk New to reduce churn.
 

Found this useful?

All lessons, notebooks and datasets here are free. If they helped you, a coffee keeps new lessons coming.