Mathew K AnalyticsLesson 9 · Market Research Analytics in Python
Introduction to Pandas for Market Research: Master Data Analysis with Python
Learn how Pandas helps analyze real market research datasets. Understand customer survey and behavioral data for better decisions. Produce insights that…
- CourseMarket Research Analytics in Python
- Lesson9 of 56
- Video21 min
- FormatJupyter notebook · 23 code cells
What you'll learn
- Understanding Market Research Data and Common Pitfalls
- Beginner Example 1: Counting Churned Customers
- Beginner Example 2: Average Tenure of Customers
- Beginner Example 3: Frequency of Each Contract Type
- Intermediate Example 1: Customer Response Rate
- Intermediate Example 2: Average Balance by Marital Status
- Intermediate Example 3: How Many Customers Have Loans?
- Advanced Example 1: Highest Revenue Products
Data
No separate download needed — the notebook creates or downloads everything it uses.
📓 Full notebook
Download .ipynbIntroduction to Pandas for Market Research#
- Learn how Pandas helps analyze real market research datasets.
- Understand customer survey and behavioral data for better decisions.
- Produce insights that inform product, marketing, and service strategy.
- These practical steps let you gain value from raw survey results.
import pandas as pd
import numpy as np
import openml
import warnings
warnings.filterwarnings('ignore')
Understanding Market Research Data and Common Pitfalls#
- Market research data includes surveys, campaigns, and customer behaviors.
- Columns may represent responses, demographics, and feedback.
- Each row is a survey response, purchase event, or customer record.
- Beginners may misinterpret missing values or mishandle categories.
- Always check data types and understand what each field means.
dataset = openml.datasets.get_dataset(42178)
df, _, _, _ = dataset.get_data(dataset_format='dataframe')
print(df.shape)
print(df.head(3))
Beginner Example 1: Counting Churned Customers#
- Business often wants to know how many customers are leaving.
- We will count how many customers have churned based on survey data.
churn_counts = df['Churn'].value_counts()
print(churn_counts)
Beginner Example 2: Average Tenure of Customers#
- Tenure tells how long customers stay with the company.
- Finding the average can help spot trends in loyalty.
avg_tenure = df['tenure'].mean()
print('Average tenure (months):', round(avg_tenure,1))
Beginner Example 3: Frequency of Each Contract Type#
- See how many customers are on month-to-month, one year, or two year contracts.
- This helps spot popular contract types in the survey.
contract_counts = df['Contract'].value_counts()
print(contract_counts)
dataset = openml.datasets.get_dataset(1461)
df_marketing, _, _, _ = dataset.get_data(dataset_format='dataframe')
df_marketing.columns = ['age','job','marital','education','default','balance','housing','loan','contact','day','month','duration','campaign','pdays','previous','poutcome','response']
print(df_marketing.shape)
print(df_marketing.head(3))
Intermediate Example 1: Customer Response Rate#
- Knowing the overall campaign response rate is a key marketing metric.
- We will calculate the percent of customers who responded 'yes'.
total = len(df_marketing)
yes_count = (df_marketing['response'] == 'yes').sum()
response_rate = yes_count / total * 100
print(f'Response rate: {response_rate:.2f}%')
Intermediate Example 2: Average Balance by Marital Status#
- Segmenting by marital status can reveal spending behavior patterns.
- We will break down average account balance for each status group.
avg_balance_by_marital = df_marketing.groupby('marital')['balance'].mean()
print(avg_balance_by_marital)
Intermediate Example 3: How Many Customers Have Loans?#
- Loan information helps assess customer credit risk for new campaigns.
- We will count customers who have at least one loan.
loan_holders = df_marketing[df_marketing['loan']=='yes'].shape[0]
print(f'Number of customers with loans: {loan_holders}')
url = 'https://archive.ics.uci.edu/ml/machine-learning-databases/00502/online_retail_II.xlsx'
df_retail = pd.read_excel(url, sheet_name='Year 2010-2011')
df_retail['InvoiceDate'] = pd.to_datetime(df_retail['InvoiceDate'])
print(df_retail.shape)
print(df_retail.head(3))
Advanced Example 1: Highest Revenue Products#
- Companies want to know which products drive the most revenue.
- We will aggregate total sales by product and sort the top sellers.
df_retail['Revenue'] = df_retail['Quantity'] * df_retail['Price']
top_products = df_retail.groupby('Description')['Revenue'].sum().sort_values(ascending=False).head(5)
print(top_products)
Advanced Example 2: Cohort Analysis of Customer Retention#
- Cohort analysis tracks customer groups over time.
- We will simulate simple monthly user activity to spot retention trends.
np.random.seed(0)
dates = pd.date_range('2021-01-01', periods=24, freq='M')
cohort_df = pd.DataFrame({'CustomerID': np.random.randint(1000,2000,len(dates)),
'Signup_Month': dates,
'Active_Users': np.random.randint(50,300,len(dates))})
print(cohort_df.head(3))
import matplotlib.pyplot as plt
plt.plot(cohort_df['Signup_Month'], cohort_df['Active_Users'], marker='o')
plt.title('Customer Cohort Active Users Over Time')
plt.xlabel('Signup Month')
plt.ylabel('Active Users')
plt.xticks(rotation=45)
plt.tight_layout()
plt.show()
Error Handling: Spotting Missing Survey Responses#
- Missing values can bias results if you do not check for them.
- Always count missing data before analysis.
print(df.isnull().sum())
Error Handling: Incorrect Grouping in Market Analysis#
- A common mistake is grouping on the wrong field and misreading results.
- Always triple-check your groupby before making business recommendations.
# Intentionally incorrect grouping (for teaching!)
bad_group = df.groupby('Partner')['MonthlyCharges'].mean()
print(bad_group)
Error Handling: Misreading NPS and Likert Scale Data#
- NPS and Likert scores must be interpreted using clear business rules.
- 0 is not always the worst and 10 is not always the bestdefinitions matter.
np.random.seed(42)
nps_df = pd.DataFrame({
'CustomerID': range(1,501),
'Age': np.random.randint(18,70,500),
'Region': np.random.choice(['North','South','East','West'],500),
'NPS_Score': np.random.randint(0,11,500)
})
print(nps_df.head(3))
nps_df['Type'] = np.where(nps_df['NPS_Score'] >= 9, 'Promoter',
np.where(nps_df['NPS_Score'] <= 6, 'Detractor', 'Passive'))
print(nps_df['Type'].value_counts())
Best Practices: Customer Segmentation Using Pandas#
- Divide your customers into useful groups: by age, region, or spending.
- Segmentation helps find target audiences for campaigns.
nps_df['AgeGroup'] = pd.cut(nps_df['Age'], bins=[17,29,49,69], labels=['Young','Middle','Senior'])
print(nps_df.groupby('AgeGroup')['NPS_Score'].mean())
Best Practices: Cross-Tabulation in Survey Results#
- Cross-tabulation lets you compare two categorical variables, such as region and NPS type.
- This reveals important relationships for marketing teams.
crosstab = pd.crosstab(nps_df['Region'], nps_df['Type'])
print(crosstab)
Best Practices: Score Construction for Survey Analytics#
- Index scores help make survey data meaningful for business reports.
- Let us create a 'Satisfaction Index' as a new metric.
nps_df['Satisfaction_Index'] = (nps_df['NPS_Score'] / 10) * 100
print(nps_df[['CustomerID','NPS_Score','Satisfaction_Index']].head(3))
Best Practices: Trend Analysis on Market Data#
- Spotting customer trends over time is vital for business direction.
- Track how campaign responses change each month.
monthly_trend = df_marketing.groupby('month')['response'].value_counts().unstack().fillna(0)
print(monthly_trend.head())
End-to-End Market Research: From Survey to Insight#
- We will take customer satisfaction survey data and summarize the main driver of churn.
- The analysis will lead to a practical business recommendation.
churn_by_contract = df.groupby('Contract')['Churn'].value_counts(normalize=True).unstack().fillna(0) * 100
print(churn_by_contract)
Found this useful?
All lessons, notebooks and datasets here are free. If they helped you, a coffee keeps new lessons coming.
Related lessons
14 minLesson 6
Master Data Cleaning and Preparation with Pandas in Python for Effective Bible Study Analysis
21 minLesson 44
Efficient Data Analysis with Pandas: Master Method Chaining & Pipelines in Python
12 minLesson 17
Resampling & Aggregation in Pandas: Master Time Series Data Analysis with Python
13 minLesson 10