Mathew K Analytics

Lesson 17 · Python For Time Series

Resampling & Aggregation in Pandas: Master Time Series Data Analysis with Python

This lesson will help you master resampling and aggregation using pandas. We will use real-world time series data, like sales or weather. You will learn to…

⬇ Download notebookOpen in Colab ↗

What you'll learn

Data

No separate download needed — the notebook creates or downloads everything it uses.

📓 Full notebook

Download .ipynb
 

Resampling and Aggregation in Python with Pandas#

This lesson will help you master resampling and aggregation using pandas.

We will use real-world time series data, like sales or weather.

You will learn to group data by time, summarize trends, and pull out key patterns.

Let us get started!

import warnings; warnings.filterwarnings("ignore")

# Let us set up our pandas environment
import pandas as pd
import matplotlib.pyplot as plt

What is Resampling?#

Resampling lets you change the time step of your data.

You can group by months, weeks, years, or even hours.

This helps you see patterns and trends that may be hidden when looking at raw data.

# Data setup: Load airline passengers dataset
url = "https://raw.githubusercontent.com/jbrownlee/Datasets/master/airline-passengers.csv"
df = pd.read_csv(url)

# Show the size and first few rows
print("Shape:", df.shape)
df.head()
Shape: (144, 2)
Month Passengers
0 1949-01 112
1 1949-02 118
2 1949-03 132
3 1949-04 129
4 1949-05 121
# Plot passenger counts over time
plt.figure(figsize=(10,4))
plt.plot(df['Month'], df['Passengers'])
plt.title("Airline Passengers Over Time")
plt.xlabel("Month")
plt.ylabel("Number of Passengers")
plt.xticks(rotation=45)
plt.tight_layout()
plt.show()
No description has been provided for this image

Why Aggregate Data?#

Aggregation means combining several numbers into a summary.

For example, you can take the monthly numbers and compute the yearly total.

This helps you see the big picture and compare different time intervals.

# Make Month the index and parse it as a date
df['Month'] = pd.to_datetime(df['Month'])
df = df.set_index('Month')
df.head()
Passengers
Month
1949-01-01 112
1949-02-01 118
1949-03-01 132
1949-04-01 129
1949-05-01 121
# Resample to yearly totals
yearly = df.resample('Y').sum()
yearly
Passengers
Month
1949-12-31 1520
1950-12-31 1676
1951-12-31 2042
1952-12-31 2364
1953-12-31 2700
1954-12-31 2867
1955-12-31 3408
1956-12-31 3939
1957-12-31 4421
1958-12-31 4572
1959-12-31 5140
1960-12-31 5714
# Plot the yearly totals
plt.figure(figsize=(8,3))
plt.bar(yearly.index.year, yearly['Passengers'])
plt.title("Total Airline Passengers per Year")
plt.xlabel("Year")
plt.ylabel("Yearly Total Passengers")
plt.show()
No description has been provided for this image
# Resample with mean for yearly averages
yearly_avg = df.resample('Y').mean()
yearly_avg
Passengers
Month
1949-12-31 126.666667
1950-12-31 139.666667
1951-12-31 170.166667
1952-12-31 197.000000
1953-12-31 225.000000
1954-12-31 238.916667
1955-12-31 284.000000
1956-12-31 328.250000
1957-12-31 368.416667
1958-12-31 381.000000
1959-12-31 428.333333
1960-12-31 476.166667

Exploring Different Time Windows#

You can group by different periods, not just years.

Try resampling by quarter (every three months) or by each month.

This shows seasonal trends or cycles in the data.

# Quarterly totals
quarterly = df.resample('Q').sum()
quarterly.head()
Passengers
Month
1949-03-31 362
1949-06-30 385
1949-09-30 432
1949-12-31 341
1950-03-31 382
# Find the busiest month of all time
max_month = df['Passengers'].idxmax()
max_value = df['Passengers'].max()
print(f"Busiest month: {max_month.strftime('%Y-%m')} with {max_value} passengers")
Busiest month: 1960-07 with 622 passengers
# Aggregating with multiple functions
agg = df.resample('Y').agg(['sum', 'mean', 'max'])
agg
Passengers
sum mean max
Month
1949-12-31 1520 126.666667 148
1950-12-31 1676 139.666667 170
1951-12-31 2042 170.166667 199
1952-12-31 2364 197.000000 242
1953-12-31 2700 225.000000 272
1954-12-31 2867 238.916667 302
1955-12-31 3408 284.000000 364
1956-12-31 3939 328.250000 413
1957-12-31 4421 368.416667 467
1958-12-31 4572 381.000000 505
1959-12-31 5140 428.333333 559
1960-12-31 5714 476.166667 622
# Resample and fill missing data
monthly_all = df.resample('M').sum()
monthly_all = monthly_all.reindex(pd.date_range(monthly_all.index.min(), monthly_all.index.max(), freq='M'))
monthly_all = monthly_all.fillna(method='ffill')
monthly_all.head()
Passengers
1949-01-31 112
1949-02-28 118
1949-03-31 132
1949-04-30 129
1949-05-31 121
# Rolling mean: smooth out ups and downs
df['RollingMean_6'] = df['Passengers'].rolling(window=6).mean()
df[['Passengers','RollingMean_6']].plot(figsize=(10,5))
plt.title('Passenger Trend (6-month Moving Average)')
plt.show()
No description has been provided for this image

Real-World Example: Quarterly Sales Figures#

Grouping monthly data into quarters can help businesses see seasonal demand.

Resampling can help with planning and setting goals for future sales.

# Apply a custom aggregation: percent change each year
pct_change = yearly['Passengers'].pct_change() * 100
pct_change = pct_change.round(2)
print(pct_change)
Month
1949-12-31      NaN
1950-12-31    10.26
1951-12-31    21.84
1952-12-31    15.77
1953-12-31    14.21
1954-12-31     6.19
1955-12-31    18.87
1956-12-31    15.58
1957-12-31    12.24
1958-12-31     3.42
1959-12-31    12.42
1960-12-31    11.17
Freq: YE-DEC, Name: Passengers, dtype: float64
# Mini-project part 1: Find the year with most new passengers
diffs = yearly['Passengers'].diff()
year = diffs.idxmax().year
print(f"Year with most new passengers: {year}")
Year with most new passengers: 1960
# Mini-project part 2: Classify years as 'Growth' or 'Drop'
yearly['Change'] = yearly['Passengers'].diff()
yearly['Category'] = yearly['Change'].apply(lambda x: 'Growth' if x>0 else 'Drop')
print(yearly[['Passengers','Category']])
            Passengers Category
Month                          
1949-12-31        1520     Drop
1950-12-31        1676   Growth
1951-12-31        2042   Growth
1952-12-31        2364   Growth
1953-12-31        2700   Growth
1954-12-31        2867   Growth
1955-12-31        3408   Growth
1956-12-31        3939   Growth
1957-12-31        4421   Growth
1958-12-31        4572   Growth
1959-12-31        5140   Growth
1960-12-31        5714   Growth
# Troubleshooting: Why did resample fail?
try:
    df_reset = df.reset_index()
    df_reset['FakeDate'] = df_reset['Passengers'].astype(str)
    pd.to_datetime(df_reset['FakeDate']).resample('Y')
except Exception as e:
    print("Error:", e)
    
Error: Given date string "112" not likely a datetime, at position 0
# Extra: Resample and plot only the summer months
summer = df[df.index.month.isin([6,7,8])]
summer_sum = summer.resample('Y').sum()
summer_sum.plot(kind='bar', figsize=(7,3))
plt.title('Total Passengers each Summer')
plt.show()
No description has been provided for this image

Challenge: Your Turn!#

  • Try resampling by different periods: how about weeks?

  • What happens if you use 'min' or 'max' in your aggregation?

  • Can you create a moving median as well as a mean?

Experiment with these techniques using your own data if you have some.

Recap: What You Have Learned#

  • How to load and view time series data with pandas
  • The basic idea of resampling
  • How to group and aggregate by different periods
  • Creating plots to spot trends
  • Handling missing data and smoothing with rolling statistics

These skills help you summarize, explain, and forecast real-world data.

Thanks for Learning!#

If you enjoyed this lesson, subscribe to the channel and leave a comment.

Try these techniques on your work or school projects.

Happy coding!

Found this useful?

All lessons, notebooks and datasets here are free. If they helped you, a coffee keeps new lessons coming.