Lesson 23 · Market Research Analytics in Python
Cross Tabulation and Segment Comparisons in Python for Market Research Analysis
In this lesson, we will use real survey data to compare customer segments and find valuable patterns. Businesses need to understand how different customer…
- CourseMarket Research Analytics in Python
- Lesson23 of 56
- Video28 min
- FormatJupyter notebook · 27 code cells
What you'll learn
Data
No separate download needed — the notebook creates or downloads everything it uses.
📓 Full notebook
Download .ipynbCross-Tabulation and Segment Comparisons in Market Research#
- In this lesson, we will use real survey data to compare customer segments and find valuable patterns.
- Businesses need to understand how different customer groups behave and feel to make informed decisions.
- You will learn how to create cross-tabulations, compare segments, and draw actionable insights from the results.
import pandas as pd
import numpy as np
import openml
import warnings
warnings.filterwarnings('ignore')
Core Concepts: What Is Cross-Tabulation and Segment Comparisons?#
- Market researchers segment customers to compare behaviors, satisfaction, and responses.
- Cross-tabulation is a core method: it groups data by two or more variables to observe interactions.
- Customer survey data often includes demographics, responses, and scores (like satisfaction or NPS).
- Beginners sometimes treat survey scores as continuous when they are ordinal, or forget to handle missing values.
- Segment comparisons must use correct groupings, and always interpret results with business meaning in mind.
# Beginner Example 1: Load a real customer satisfaction survey dataset
dataset = openml.datasets.get_dataset(42178)
df, _, _, _ = dataset.get_data(dataset_format='dataframe')
print(df.shape)
print(df.head(3))
# Beginner Example 2: Frequency count for a categorical segment
gender_counts = df['gender'].value_counts(dropna=False)
print(gender_counts)
# Beginner Example 3: Create a simple cross-tab (gender vs Churn)
ct = pd.crosstab(df['gender'], df['Churn'])
print(ct)
# Beginner Example 4: Cross-tab with column percentages
ct_percent = pd.crosstab(df['gender'], df['Churn'], normalize='columns')
print(np.round(ct_percent * 100, 2))
# Beginner Example 5: Cross-tab on SeniorCitizen and Contract type
contract_senior_ct = pd.crosstab(df['SeniorCitizen'], df['Contract'])
print(contract_senior_ct)
# Beginner Example 6: Handling missing values in categorical cross-tabs
missing_dependents = df['Dependents'].isna().sum()
print('Missing values in Dependents:', missing_dependents)
# Intermediate Example 1: Segmenting by Customer Tenure
bins = [0, 12, 36, 72]
labels = ['<1 yr', '1-3 yrs', '3-6 yrs']
df['TenureGroup'] = pd.cut(df['tenure'], bins=bins, labels=labels, right=True)
print(df[['tenure', 'TenureGroup']].head(5))
# Intermediate Example 2: Cross-tab with three variables (Contract, Churn, and SeniorCitizen)
ct3d = pd.crosstab([df['Contract'], df['SeniorCitizen']], df['Churn'])
print(ct3d)
# Intermediate Example 3: Segment comparison using groupby and mean
avg_monthly_charges = df.groupby('Contract')['MonthlyCharges'].mean()
print(avg_monthly_charges)
# Intermediate Example 4: Cross-tab with Marketing Campaign Dataset
ds = openml.datasets.get_dataset(1461)
mk_df, _, _, _ = ds.get_data(dataset_format='dataframe')
mk_df.columns = ['age','job','marital','education','default','balance','housing','loan','contact','day','month','duration','campaign','pdays','previous','poutcome','response']
marital_response_ct = pd.crosstab(mk_df['marital'], mk_df['response'])
print(marital_response_ct)
# Intermediate Example 5: Normalized response rates by marital status
marital_response_rate = pd.crosstab(mk_df['marital'], mk_df['response'], normalize='index')
print(np.round(marital_response_rate * 100, 1))
# Intermediate Example 6: Multiple segment crosstab - education and response
education_response_ct = pd.crosstab([mk_df['education'], mk_df['marital']], mk_df['response'])
print(education_response_ct)
# Advanced Example 1: Segmentation on NPS survey data
np.random.seed(42)
nps_df = pd.DataFrame({
'CustomerID': range(1,501),
'Age': np.random.randint(18,70,500),
'Region': np.random.choice(['North','South','East','West'],500),
'NPS_Score': np.random.randint(0,11,500)
})
nps_df['NPS_Category'] = pd.cut(nps_df['NPS_Score'], bins=[-1,6,8,10], labels=['Detractor','Passive','Promoter'])
print(nps_df[['NPS_Score','NPS_Category','Region']].head(5))
# Advanced Example 2: Cross-tabulate NPS Category by Region
nps_ct = pd.crosstab(nps_df['Region'], nps_df['NPS_Category'], normalize='index')
print(np.round(nps_ct * 100, 1))
# Advanced Example 3: Segment-specific mean age of promoters
mean_age_promoters = nps_df[nps_df['NPS_Category'] == 'Promoter'].groupby('Region')['Age'].mean()
print(mean_age_promoters)
# Advanced Example 4: Multi-way crosstab with survey and transaction data
retail_url = 'https://archive.ics.uci.edu/ml/machine-learning-databases/00502/online_retail_II.xlsx'
online_df = pd.read_excel(retail_url, sheet_name='Year 2010-2011')
online_df['InvoiceDate'] = pd.to_datetime(online_df['InvoiceDate'])
online_df['YearMonth'] = online_df['InvoiceDate'].dt.to_period('M')
ct_country_month = pd.crosstab(online_df['Country'], online_df['YearMonth'])
print(ct_country_month.head())
# Advanced Example 5: Cross-tab as a share of row total
row_share = pd.crosstab(df['Contract'], df['Churn'], normalize='index')
print(np.round(row_share * 100, 1))
# Advanced Example 6: Exporting a cross-tab to Excel for stakeholder sharing
ct_export = pd.crosstab(df['Contract'], df['Churn'])
ct_export.to_excel('contract_churn_crosstab.xlsx')
print('Cross-tab saved as contract_churn_crosstab.xlsx')
# Error Handling Example 1: Handling missing responses safely using a copy
temp_df = mk_df.copy()
temp_df.loc[0, 'response'] = np.nan
temp_df['response'] = (
temp_df['response']
.astype('object')
.fillna('Unknown')
.astype(str)
)
ct_error = pd.crosstab(temp_df['education'], temp_df['response'])
print(ct_error)
# Error Handling Example 2: Double-check groupby with wrong column
try:
wrong_group = df.groupby('Payment')['MonthlyCharges'].mean()
except KeyError as e:
print('Error:', e)
print('Tip: Check your column names. Use df.columns to see available columns.')
# Error Handling Example 3: Accidental numeric aggregation on Likert or NPS scores
try:
likert_mean = df['OnlineBackup'].mean()
print('Mean OnlineBackup score:', likert_mean)
except Exception as ex:
print('Error:', ex)
print('Tip: Always check if the survey column is truly numeric or categorical!')
# Best Practice 1: Create a segment index from crosstab proportions
ct_index = pd.crosstab(df['SeniorCitizen'], df['Churn'], normalize='index')
ct_index['Churn_Index'] = ct_index['Yes'] / ct_index['No']
print(np.round(ct_index[['Churn_Index']], 2))
# Best Practice 2: Visualizing segment comparisons for stakeholders
import matplotlib.pyplot as plt
row_share.plot(kind='bar', stacked=True, color=['green','red'])
plt.title('Churn Rate by Contract Type')
plt.xlabel('Contract Type')
plt.ylabel('Proportion')
plt.legend(title='Churn')
plt.tight_layout()
plt.savefig('churn_barplot.png')
plt.close()
print('Bar plot saved as churn_barplot.png')
# Best Practice 3: Trend analysis with cross-tab time series
monthly_churn = pd.crosstab(df['Contract'], df['Churn'])
if 'InvoiceDate' in df.columns:
df['YearMonth'] = df['InvoiceDate'].dt.to_period('M')
churn_trends = pd.crosstab(df['YearMonth'], df['Churn'])
print(churn_trends.head())
else:
print('InvoiceDate not available in this survey data.')
End-to-End Example: From Raw Data to Customer Insight#
- We will load NPS survey results, cross-tabulate to compare loyalty by region, and give an actionable recommendation.
- Segmenting and cross-tabbing lets us focus business resources where they matter most.
- This is how market research informs better products and services.
# Load and segment NPS data for a business recommendation
nps_summary = pd.crosstab(nps_df['Region'], nps_df['NPS_Category'], normalize='index')
print(np.round(nps_summary * 100, 1))
top_region = nps_summary['Promoter'].idxmax()
low_region = nps_summary['Detractor'].idxmax()
print('Highest loyalty region:', top_region)
print('Most at-risk region:', low_region)
Found this useful?
All lessons, notebooks and datasets here are free. If they helped you, a coffee keeps new lessons coming.



