Mathew K Analytics

Lesson 32 · Mastering Pandas

Mastering Data Reshaping in Pandas with Melt, Stack, and Unstack Functions

In this lesson, we will explore powerful tools for reshaping your data in pandas. You will learn why and how to use melt, stack, and unstack to reorganize…

⬇ Download notebookOpen in Colab ↗

What you'll learn

Data

No separate download needed — the notebook creates or downloads everything it uses.

📓 Full notebook

Download .ipynb

Reshaping Data Using Pandas: melt, stack, and unstack#

In this lesson, we will explore powerful tools for reshaping your data in pandas. You will learn why and how to use melt, stack, and unstack to reorganize your DataFrames. These techniques help you prepare your data for deeper analysis and visualization.

import warnings
import numpy as np
import pandas as pd
np.random.seed(42)
warnings.filterwarnings('ignore')
# Data setup (Airline Passenger Flights Dataset)
import seaborn as sns
df = sns.load_dataset('flights')
print(df.shape)
print(df.head(3))
(144, 3)
   year month  passengers
0  1949   Jan         112
1  1949   Feb         118
2  1949   Mar         132

Why reshape data?#

Most datasets arrive in a shape that is convenient for recording or collecting, not for analysis. Reshaping lets you swivel, pivot, or flatten your data for tasks like time series, machine learning, or dashboards. Lets see how the Flights dataset can be re-organized for different questions.

# Preview the original DataFrame structure
df.head()
year month passengers
0 1949 Jan 112
1 1949 Feb 118
2 1949 Mar 132
3 1949 Apr 129
4 1949 May 121

What are pandas melt, stack, and unstack?#

  • melt: Turns wide tables into long ones (good for plotting and machine learning).
  • stack: Pivots columns into rows in a hierarchical index.
  • unstack: The opposite of stack; pivots rows back to columns.

We will use each to reshape our Flights data.

# Pivot to wide format: years become columns
df_wide = df.pivot(index='month', columns='year', values='passengers')
print(df_wide.shape)
df_wide.head()
(12, 12)
year 1949 1950 1951 1952 1953 1954 1955 1956 1957 1958 1959 1960
month
Jan 112 115 145 171 196 204 242 284 315 340 360 417
Feb 118 126 150 180 196 188 233 277 301 318 342 391
Mar 132 141 178 193 236 235 267 317 356 362 406 419
Apr 129 135 163 181 235 227 269 313 348 348 396 461
May 121 125 172 183 229 234 270 318 355 363 420 472
# Use melt to turn the wide DataFrame back into long format
df_long = df_wide.reset_index().melt(id_vars='month', var_name='year', value_name='passengers')
print(df_long.shape)
df_long.head()
(144, 3)
month year passengers
0 Jan 1949 112
1 Feb 1949 118
2 Mar 1949 132
3 Apr 1949 129
4 May 1949 121

What just happened?#

  • pivot widened the data so each year is a column.
  • melt reversed it, making a long tidy table again.

This is often needed for plotting software or machine learning models, which expect a long format.

# Stack example: move columns into indexed rows
stacked = df_wide.stack()
print(stacked.shape)
stacked.head(12)
(144,)
month  year
Jan    1949    112
       1950    115
       1951    145
       1952    171
       1953    196
       1954    204
       1955    242
       1956    284
       1957    315
       1958    340
       1959    360
       1960    417
dtype: int64
# Notice the MultiIndex structure after stacking
print(stacked.index.names)
['month', 'year']
# Unstack example: convert the MultiIndex Series back to DataFrame
unstacked = stacked.unstack()
print(unstacked.shape)
unstacked.head()
(12, 12)
year 1949 1950 1951 1952 1953 1954 1955 1956 1957 1958 1959 1960
month
Jan 112 115 145 171 196 204 242 284 315 340 360 417
Feb 118 126 150 180 196 188 233 277 301 318 342 391
Mar 132 141 178 193 236 235 267 317 356 362 406 419
Apr 129 135 163 181 235 227 269 313 348 348 396 461
May 121 125 172 183 229 234 270 318 355 363 420 472

Quick recap so far#

  • Pivot: Makes wide tables by moving values to columns.
  • Melt: Returns it to long format with one measurement per row.
  • Stack/Unstack: Toggle between columns and levels of row index (great for multilevel data).
# Visualize: compare original and melted forms
import matplotlib.pyplot as plt
plt.figure(figsize=(8,4))
plt.plot(df['passengers'], label='Original/order')
plt.plot(df_long.sort_values(['year','month'])['passengers'].values, '--', label='Melted/sorted')
plt.title('Passengers Over Time (Original vs Melted)')
plt.legend()
plt.show()
No description has been provided for this image
# Challenge: creating a tidy table for easy aggregation
df_tidy = df.copy()
df_tidy['date'] = pd.to_datetime(df_tidy['year'].astype(str) + '-' + df_tidy['month'].astype(str) + '-01')
df_tidy = df_tidy.sort_values('date')
df_tidy.set_index('date', inplace=True)
df_tidy.head()
year month passengers
date
1949-01-01 1949 Jan 112
1949-02-01 1949 Feb 118
1949-03-01 1949 Mar 132
1949-04-01 1949 Apr 129
1949-05-01 1949 May 121
# Stack and unstack with multi-level columns: creating multi-index columns
multi = df.pivot_table(index='month', columns='year', values='passengers', aggfunc='sum')
multi.head()
year 1949 1950 1951 1952 1953 1954 1955 1956 1957 1958 1959 1960
month
Jan 112 115 145 171 196 204 242 284 315 340 360 417
Feb 118 126 150 180 196 188 233 277 301 318 342 391
Mar 132 141 178 193 236 235 267 317 356 362 406 419
Apr 129 135 163 181 235 227 269 313 348 348 396 461
May 121 125 172 183 229 234 270 318 355 363 420 472
# Stack across columns: get a long Series from the table with multi-level columns
long_series = multi.stack()
print(long_series.shape)
long_series.head(10)
(144,)
month  year
Jan    1949    112
       1950    115
       1951    145
       1952    171
       1953    196
       1954    204
       1955    242
       1956    284
       1957    315
       1958    340
dtype: int64
# Unstack to return the Series to a DataFrame
back_to_df = long_series.unstack()
back_to_df.head()
year 1949 1950 1951 1952 1953 1954 1955 1956 1957 1958 1959 1960
month
Jan 112 115 145 171 196 204 242 284 315 340 360 417
Feb 118 126 150 180 196 188 233 277 301 318 342 391
Mar 132 141 178 193 236 235 267 317 356 362 406 419
Apr 129 135 163 181 235 227 269 313 348 348 396 461
May 121 125 172 183 229 234 270 318 355 363 420 472

The Power of Reshaping: Real-World Use Cases#

  • Reporting: Daily to monthly summaries.
  • Visualization: Prepare long-form data for line plots or heatmaps.
  • Machine Learning: Convert tables to an 'observation per row' format.

Mastering these tools will save you hours in data analysis.

# Troubleshooting: Common issues with stack/unstack/melt
try:
    df_broken = df_wide.reset_index().melt(id_vars='wrong_column')
except Exception as e:
    print('Error:', e)
    
Error: "The following id_vars or value_vars are not present in the DataFrame: ['wrong_column']"
# Mini-project: Reshape and aggregate flights data
monthly_avg = df.pivot_table(index='month', values='passengers', aggfunc='mean')
print(monthly_avg)
max_passenger_month = monthly_avg['passengers'].idxmax()
print('Month with highest average passengers:', max_passenger_month)
       passengers
month            
Jan    241.750000
Feb    235.000000
Mar    270.166667
Apr    267.083333
May    271.833333
Jun    311.666667
Jul    351.333333
Aug    351.083333
Sep    302.416667
Oct    266.583333
Nov    232.833333
Dec    261.833333
Month with highest average passengers: Jul
# Your turn! Try reshaping a DataFrame of your own (interactive)
col = input('Type one of these to try melt or stack: month, year, passengers: ')
if col.strip().lower() == 'month':
    melted = df.melt(id_vars='month')
    print(melted.head())
elif col.strip().lower() == 'year':
    melted = df.melt(id_vars='year')
    print(melted.head())
elif col.strip().lower() == 'passengers':
    stacked = df.set_index(['month','year']).stack()
    print(stacked.head())
else:
    print('Try again with month, year, or passengers.')
    
  month variable  value
0   Jan     year   1949
1   Feb     year   1949
2   Mar     year   1949
3   Apr     year   1949
4   May     year   1949

Recap#

You have learned to:

  • Reorganize tables with melt, stack, and unstack.
  • Tidy your data for aggregation and visualization.
  • Handle and avoid common reshaping pitfalls.

Try using these steps in your next analysis to gain better insights, faster.

Thanks for learning with us!#

Practice these techniques on new datasets. For more pandas tutorials, subscribe to our channel and leave a comment below.

Happy coding and keep learning!

Found this useful?

All lessons, notebooks and datasets here are free. If they helped you, a coffee keeps new lessons coming.