Lesson 17 · Python For Time Series
Resampling & Aggregation in Pandas: Master Time Series Data Analysis with Python
This lesson will help you master resampling and aggregation using pandas. We will use real-world time series data, like sales or weather. You will learn to…
- CoursePython For Time Series
- Lesson17 of 30
- Video12 min
- FormatJupyter notebook · 18 code cells
What you'll learn
Data
No separate download needed — the notebook creates or downloads everything it uses.
📓 Full notebook
Download .ipynb
Resampling and Aggregation in Python with Pandas#
This lesson will help you master resampling and aggregation using pandas.
We will use real-world time series data, like sales or weather.
You will learn to group data by time, summarize trends, and pull out key patterns.
Let us get started!
import warnings; warnings.filterwarnings("ignore")
# Let us set up our pandas environment
import pandas as pd
import matplotlib.pyplot as plt
What is Resampling?#
Resampling lets you change the time step of your data.
You can group by months, weeks, years, or even hours.
This helps you see patterns and trends that may be hidden when looking at raw data.
# Data setup: Load airline passengers dataset
url = "https://raw.githubusercontent.com/jbrownlee/Datasets/master/airline-passengers.csv"
df = pd.read_csv(url)
# Show the size and first few rows
print("Shape:", df.shape)
df.head()
# Plot passenger counts over time
plt.figure(figsize=(10,4))
plt.plot(df['Month'], df['Passengers'])
plt.title("Airline Passengers Over Time")
plt.xlabel("Month")
plt.ylabel("Number of Passengers")
plt.xticks(rotation=45)
plt.tight_layout()
plt.show()
Why Aggregate Data?#
Aggregation means combining several numbers into a summary.
For example, you can take the monthly numbers and compute the yearly total.
This helps you see the big picture and compare different time intervals.
# Make Month the index and parse it as a date
df['Month'] = pd.to_datetime(df['Month'])
df = df.set_index('Month')
df.head()
# Resample to yearly totals
yearly = df.resample('Y').sum()
yearly
# Plot the yearly totals
plt.figure(figsize=(8,3))
plt.bar(yearly.index.year, yearly['Passengers'])
plt.title("Total Airline Passengers per Year")
plt.xlabel("Year")
plt.ylabel("Yearly Total Passengers")
plt.show()
# Resample with mean for yearly averages
yearly_avg = df.resample('Y').mean()
yearly_avg
Exploring Different Time Windows#
You can group by different periods, not just years.
Try resampling by quarter (every three months) or by each month.
This shows seasonal trends or cycles in the data.
# Quarterly totals
quarterly = df.resample('Q').sum()
quarterly.head()
# Find the busiest month of all time
max_month = df['Passengers'].idxmax()
max_value = df['Passengers'].max()
print(f"Busiest month: {max_month.strftime('%Y-%m')} with {max_value} passengers")
# Aggregating with multiple functions
agg = df.resample('Y').agg(['sum', 'mean', 'max'])
agg
# Resample and fill missing data
monthly_all = df.resample('M').sum()
monthly_all = monthly_all.reindex(pd.date_range(monthly_all.index.min(), monthly_all.index.max(), freq='M'))
monthly_all = monthly_all.fillna(method='ffill')
monthly_all.head()
# Rolling mean: smooth out ups and downs
df['RollingMean_6'] = df['Passengers'].rolling(window=6).mean()
df[['Passengers','RollingMean_6']].plot(figsize=(10,5))
plt.title('Passenger Trend (6-month Moving Average)')
plt.show()
Real-World Example: Quarterly Sales Figures#
Grouping monthly data into quarters can help businesses see seasonal demand.
Resampling can help with planning and setting goals for future sales.
# Apply a custom aggregation: percent change each year
pct_change = yearly['Passengers'].pct_change() * 100
pct_change = pct_change.round(2)
print(pct_change)
# Mini-project part 1: Find the year with most new passengers
diffs = yearly['Passengers'].diff()
year = diffs.idxmax().year
print(f"Year with most new passengers: {year}")
# Mini-project part 2: Classify years as 'Growth' or 'Drop'
yearly['Change'] = yearly['Passengers'].diff()
yearly['Category'] = yearly['Change'].apply(lambda x: 'Growth' if x>0 else 'Drop')
print(yearly[['Passengers','Category']])
# Troubleshooting: Why did resample fail?
try:
df_reset = df.reset_index()
df_reset['FakeDate'] = df_reset['Passengers'].astype(str)
pd.to_datetime(df_reset['FakeDate']).resample('Y')
except Exception as e:
print("Error:", e)
# Extra: Resample and plot only the summer months
summer = df[df.index.month.isin([6,7,8])]
summer_sum = summer.resample('Y').sum()
summer_sum.plot(kind='bar', figsize=(7,3))
plt.title('Total Passengers each Summer')
plt.show()
Challenge: Your Turn!#
Try resampling by different periods: how about weeks?
What happens if you use 'min' or 'max' in your aggregation?
Can you create a moving median as well as a mean?
Experiment with these techniques using your own data if you have some.
Recap: What You Have Learned#
- How to load and view time series data with pandas
- The basic idea of resampling
- How to group and aggregate by different periods
- Creating plots to spot trends
- Handling missing data and smoothing with rolling statistics
These skills help you summarize, explain, and forecast real-world data.
Thanks for Learning!#
If you enjoyed this lesson, subscribe to the channel and leave a comment.
Try these techniques on your work or school projects.
Happy coding!
Found this useful?
All lessons, notebooks and datasets here are free. If they helped you, a coffee keeps new lessons coming.



