Lesson 36 · Mastering Pandas
Mastering Time Series Analysis in Python: Resampling, Shifting, and Rolling Windows with Pandas
Welcome! In this lesson, you will gain hands-on practice with intermediate time-series tasks in pandas. You will use the airline passenger Flights dataset…
- CourseMastering Pandas
- Lesson36 of 44
- Video15 min
- FormatJupyter notebook · 15 code cells
What you'll learn
Data
No separate download needed — the notebook creates or downloads everything it uses.
📓 Full notebook
Download .ipynbPandas Time-Series: Resampling, Shifting, and Rolling Windows#
Welcome! In this lesson, you will gain hands-on practice with intermediate time-series tasks in pandas.
You will use the airline passenger Flights dataset to learn about:
- Resampling data to different time periods
- Shifting values for comparisons
- Calculating rolling window statistics
These are core skills for analyzing trends, smoothing, and detecting seasonality in real-world data.
import warnings; warnings.filterwarnings("ignore") # Suppress warnings for clean output
import numpy as np
import pandas as pd
np.random.seed(42)
# Data setup (Airline Passenger Flights Dataset)
import seaborn as sns
df = sns.load_dataset('flights')
print(df.shape)
print(df.head(3))
What does our data look like?#
Each row is a month/year and a count of airline passengers.
- 'year': the year
- 'month': the month name
- 'passengers': total airline passengers that month
For time series work, we need a pandas datetime index.
# Combine 'year' and 'month', and set as new datetime index
df['date'] = pd.to_datetime(df['year'].astype(str) + '-' + df['month'].astype(str) + '-01')
df = df.set_index('date')
print(df.head(3))
# Plotting original time series: monthly passengers
import matplotlib.pyplot as plt
df['passengers'].plot(figsize=(10, 4), title='Monthly Airline Passengers')
plt.ylabel('Passengers')
plt.xlabel('Date')
plt.show()
What is resampling?#
Resampling means changing the frequency of your data.
- Downsampling: going from higher to lower frequency (monthly to yearly)
- Upsampling: going from lower to higher frequency (monthly to daily)
You can use resampling to spot yearly or quarterly trends.
# Downsample: resample monthly data to yearly by summing passengers
annual = df['passengers'].resample('Y').sum()
print(annual.head())
# Plot the annual passenger totals
annual.plot(kind='bar', figsize=(7,4), title='Yearly Total Passengers')
plt.ylabel('Passengers')
plt.xlabel('Year')
plt.tight_layout()
plt.show()
# Upsample: move from monthly to daily (with forward fill for missing data)
daily = df['passengers'].resample('D').ffill()
print(daily[:10])
Shifting data: comparing to the past or future#
Shifting means moving values forward or backward in time.
This helps us compute things like month-over-month or year-over-year changes.
# Create a new column showing previous month's passengers (shift by 1 row)
df['previous_month'] = df['passengers'].shift(1)
print(df[['passengers', 'previous_month']].head(5))
# Calculate percent change from previous month
df['percent_change'] = df['passengers'].pct_change() * 100
print(df[['passengers', 'percent_change']].head())
# Plot percent monthly change
df['percent_change'].plot(figsize=(10,4), grid=True, title='Percent Change vs. Previous Month')
plt.ylabel('Percent change')
plt.xlabel('Date')
plt.axhline(0, color='grey', linestyle='--')
plt.show()
Rolling windows: moving averages and more#
A rolling window looks at a set of recent periods (like 3 months) to summarize local trends.
Commonly, rolling windows compute averages to smooth noisy data.
# Compute rolling 12-month average of passengers
df['rolling_mean_12'] = df['passengers'].rolling(window=12).mean()
print(df[['passengers', 'rolling_mean_12']].tail(13))
# Plot original passengers and rolling mean
plt.figure(figsize=(10,4))
plt.plot(df.index, df['passengers'], label='Monthly Passengers')
plt.plot(df.index, df['rolling_mean_12'], label='12-Month Moving Avg', linewidth=2)
plt.legend()
plt.title('Passengers vs. 12-Month Rolling Average')
plt.ylabel('Passengers')
plt.xlabel('Date')
plt.show()
# Rolling standard deviation (shows how much results vary month to month)
df['rolling_std_6'] = df['passengers'].rolling(window=6).std()
print(df[['passengers', 'rolling_std_6']].tail(8))
# Combine shifting and rolling: month-over-month change in rolling average
df['rolling_mean_diff'] = df['rolling_mean_12'].diff()
print(df[['rolling_mean_12', 'rolling_mean_diff']].tail())
Mini-Project: Highlighting key periods with rolling averages#
Let us combine what we have learned.
We will find the 6-month period that had the largest average number of passengers.
# Find rolling 6-month averages and highlight the period with the highest value
df['rolling_mean_6'] = df['passengers'].rolling(window=6).mean()
max_idx = df['rolling_mean_6'].idxmax()
max_period = df.loc[max_idx- pd.DateOffset(months=5):max_idx]
print(max_period[['passengers', 'rolling_mean_6']])
Recap: Resampling, Shifting, and Rolling Windows in Pandas#
- Resampling adjusts how often we measure our data.
- Shifting lets us compare values across time.
- Rolling windows help us smooth, spot trends, and measure volatility.
With these tools, you can handle many real-life time-series questions.
Thank you for learning with us!
Try applying rolling and shifting operations to your own datasets!
If you learned something new, please like and subscribe for more pandas tutorials on our channel.
Found this useful?
All lessons, notebooks and datasets here are free. If they helped you, a coffee keeps new lessons coming.



