Mathew K Analytics

Lesson 31 · Mastering Pandas

Master Pivot Tables & Cross-Tabulations in Python Pandas for Powerful Data Analysis

Welcome! Today we will learn how to use pivot tables and crosstabs with pandas. Pivot tables help summarize data similarly to spreadsheets. We will use the…

⬇ Download notebookOpen in Colab ↗

What you'll learn

Data

No separate download needed — the notebook creates or downloads everything it uses.

📓 Full notebook

Download .ipynb

Pivot Tables and Cross-tabulations in Pandas#

Welcome! Today we will learn how to use pivot tables and crosstabs with pandas.

Pivot tables help summarize data similarly to spreadsheets. We will use the Titanic dataset for easy-to-relate examples.

By the end of this lesson, you will:

  • Summarize datasets with pivots.
  • Compare categories using cross-tabulations.
  • Handle missing values smartly.
  • Create useful tables for real decisions.
# Suppress warnings for cleaner output
import warnings
import numpy as np
np.random.seed(42)
warnings.filterwarnings("ignore")
 
 
# Data setup (Titanic Dataset)
import pandas as pd
url = 'https://raw.githubusercontent.com/datasciencedojo/datasets/master/titanic.csv'
df = pd.read_csv(url)
print(df.shape)
print(df.head(3))
(891, 12)
   PassengerId  Survived  Pclass  \
0            1         0       3   
1            2         1       1   
2            3         1       3   

                                                Name     Sex   Age  SibSp  \
0                            Braund, Mr. Owen Harris    male  22.0      1   
1  Cumings, Mrs. John Bradley (Florence Briggs Th...  female  38.0      1   
2                             Heikkinen, Miss. Laina  female  26.0      0   

   Parch            Ticket     Fare Cabin Embarked  
0      0         A/5 21171   7.2500   NaN        S  
1      0          PC 17599  71.2833   C85        C  
2      0  STON/O2. 3101282   7.9250   NaN        S  

What is a Pivot Table?#

A pivot table reshapes data for powerful exploration.

It lets you compare values quicklylike grouping by class or gender.

# Basic pivot: mean age by passenger class
pivot_simple = df.pivot_table(values='Age', index='Pclass', aggfunc='mean')
print(pivot_simple)
              Age
Pclass           
1       38.233441
2       29.877630
3       25.140620
# Multiple aggregations: survival rate and average fare by gender
pivot_mult = df.pivot_table(values=['Survived', 'Fare'], index='Sex',
                            aggfunc={'Survived': 'mean', 'Fare': 'mean'})
print(pivot_mult)
             Fare  Survived
Sex                        
female  44.479818  0.742038
male    25.523893  0.188908

Adding Columns: Index versus Columns#

You can split data by rows (index), columns, or both. Let us add 'Pclass' as columns alongside 'Sex' as index.

# Pivot with both index and columns
pivot_grid = df.pivot_table(values='Survived', index='Sex', columns='Pclass', aggfunc='mean')
print(pivot_grid)
Pclass         1         2         3
Sex                                 
female  0.968085  0.921053  0.500000
male    0.368852  0.157407  0.135447
# Filling missing values with zeros
pivot_nas = df.pivot_table(values='Survived', index='Sex', columns='Embarked', aggfunc='mean', fill_value=0)
print(pivot_nas)
Embarked         C         Q         S
Sex                                   
female    0.876712  0.750000  0.689655
male      0.305263  0.073171  0.174603

Changing the Aggregation Function#

The 'aggfunc' argument controls how data is summarized. It could be 'mean', 'sum', 'count', 'min', 'max', or even custom.

# Counting passengers per class and gender
pivot_count = df.pivot_table(values='Name', index='Pclass', columns='Sex', aggfunc='count')
print(pivot_count)
Sex     female  male
Pclass              
1           94   122
2           76   108
3          144   347
# Custom aggregation: minimum and maximum ages by embarked port
pivot_custom = df.pivot_table(values='Age', index='Embarked', aggfunc=['min', 'max'])
print(pivot_custom)
           min   max
           Age   Age
Embarked            
C         0.42  71.0
Q         2.00  70.5
S         0.67  80.0

Introduction to Cross-tabulation#

Crosstabs act like pivot tables, but always with counts or proportions. They are great for finding relationships between two columns, such as gender and class.

# Basic crosstab: count of passengers by class and gender
ctab = pd.crosstab(index=df['Pclass'], columns=df['Sex'])
print(ctab)
Sex     female  male
Pclass              
1           94   122
2           76   108
3          144   347
# Crosstab with proportions instead of counts
ctab_prop = pd.crosstab(df['Sex'], df['Pclass'], normalize='index')
print(ctab_prop.round(2))
Pclass     1     2     3
Sex                     
female  0.30  0.24  0.46
male    0.21  0.19  0.60

Combining Pivot Tables and Crosstabs#

Pivot tables and crosstabs both summarize data. Pivots are for math like mean or sum, while crosstabs are for counts or proportions.

Choose the right tool for your question!

# Pivot table with multiple values: survival by class and embark port
pivot_combined = df.pivot_table(values='Survived', index='Pclass', columns='Embarked', aggfunc='mean')
print(pivot_combined.round(2))
Embarked     C     Q     S
Pclass                    
1         0.69  0.50  0.58
2         0.53  0.67  0.46
3         0.38  0.38  0.19
# Crosstab for family size: groups based on siblings/spouse count
ctab_family = pd.crosstab(df['SibSp'], df['Survived'])
print(ctab_family)
Survived    0    1
SibSp             
0         398  210
1          97  112
2          15   13
3          12    4
4          15    3
5           5    0
8           7    0
# Quick visualization of a pivot table
import matplotlib.pyplot as plt
pivot_grid.plot(kind='bar')
plt.title('Survival Rate by Gender and Class')
plt.xlabel('Sex')
plt.ylabel('Survival Rate')
plt.legend(title='Pclass')
plt.tight_layout()
plt.show()
No description has been provided for this image

Practice: Your Turn!#

Pick any numeric column (like 'Fare' or 'Age') and try to summarize it by two categories using a pivot table.

Next, create a crosstab for 'Parch' (number of parents/children aboard) and 'Survived'.

Share what pattern you find as a comment below if you are watching on YouTube!

Recap: Key Points from This Lesson#

  • Use pivot tables to quickly explore averages, totals, or counts by group.
  • Crosstabs are for counting or proportions between two categories.
  • Fill missing values if your table has gaps.
  • Plots bring pivots to life and reveal outliers fast.

Keep experimentingdata tells its story only if you ask!

Thanks for joining! For more pandas tutorials and real data projects, subscribe to the channel and comment with your favorite trick. See you next time!#

Found this useful?

All lessons, notebooks and datasets here are free. If they helped you, a coffee keeps new lessons coming.