Mathew K Analytics

Lesson 10 · Pandas Projects

Weather Data Analysis with Python Pandas: Real Project Walkthrough

A complete, standalone tutorial: build a synthetic year of daily weather, then clean, resample, and analyze it with pandas. No prior pandas experience…

⬇ Download notebookOpen in Colab ↗

What you'll learn

Data

No separate download needed — the notebook creates or downloads everything it uses.

📓 Full notebook

Download .ipynb

Pandas for Weather Data: A Full Year of Daily Records#

  • A complete, standalone tutorial: build a synthetic year of daily weather, then clean, resample, and analyze it with pandas.
  • No prior pandas experience needed. Let's jump straight in.

Before You Start#

  • Open a new Jupyter Notebook in VS Code and select your Python interpreter as the kernel.
  • If pandas isn't installed yet, open a terminal in VS Code and run: pip install pandas

Part 1: Building the Dataset#

import pandas as pd
import numpy as np
print(pd.__version__)
2.3.0

Generating a Synthetic Year of Weather#

rng = np.random.default_rng(seed=33)
dates = pd.date_range('2026-01-01', periods=365, freq='D')
day_of_year = np.arange(365)

seasonal_swing = 12 * np.sin(2 * np.pi * (day_of_year - 80) / 365)
avg_temp = 18 + seasonal_swing + rng.normal(0, 2.5, size=365)
avg_temp = avg_temp.round(1)
print(avg_temp[:5])
[7.2 4.9 7.8 6.5 2.5]
min_temp = (avg_temp - rng.uniform(3, 7, size=365)).round(1)
max_temp = (avg_temp + rng.uniform(3, 7, size=365)).round(1)
precipitation = np.where(rng.random(365) < 0.3, rng.exponential(6, size=365).round(1), 0.0)
humidity = np.clip(rng.normal(60, 12, size=365), 20, 100).round(0)

weather = pd.DataFrame({
    'date': dates,
    'avg_temp_c': avg_temp,
    'min_temp_c': min_temp,
    'max_temp_c': max_temp,
    'precipitation_mm': precipitation,
    'humidity_pct': humidity
})
print(weather.shape)
(365, 6)

Injecting a Few Missing Readings#

missing_idx = rng.choice(weather.index, size=8, replace=False)
weather.loc[missing_idx, 'avg_temp_c'] = np.nan
weather.to_csv('daily_weather.csv', index=False)
weather = pd.read_csv('daily_weather.csv', parse_dates=['date'])
weather.head()
date avg_temp_c min_temp_c max_temp_c precipitation_mm humidity_pct
0 2026-01-01 7.2 1.9 11.9 0.0 42.0
1 2026-01-02 4.9 0.0 11.7 0.0 48.0
2 2026-01-03 7.8 4.3 13.5 2.6 62.0
3 2026-01-04 6.5 1.3 10.0 0.0 57.0
4 2026-01-05 NaN -2.6 9.1 1.5 50.0

Part 2: First Look and Cleaning#

print(weather.shape)
weather.info()
(365, 6)
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 365 entries, 0 to 364
Data columns (total 6 columns):
 #   Column            Non-Null Count  Dtype         
---  ------            --------------  -----         
 0   date              365 non-null    datetime64[ns]
 1   avg_temp_c        357 non-null    float64       
 2   min_temp_c        365 non-null    float64       
 3   max_temp_c        365 non-null    float64       
 4   precipitation_mm  365 non-null    float64       
 5   humidity_pct      365 non-null    float64       
dtypes: datetime64[ns](1), float64(5)
memory usage: 17.2 KB
weather.isna().sum()
date                0
avg_temp_c          8
min_temp_c          0
max_temp_c          0
precipitation_mm    0
humidity_pct        0
dtype: int64

Filling Missing Temperatures#

weather = weather.sort_values('date').reset_index(drop=True)
weather['avg_temp_c'] = weather['avg_temp_c'].interpolate()
weather['avg_temp_c'].isna().sum()
np.int64(0)

Part 3: Working with Dates#

weather = weather.set_index('date')
weather.loc['2026-07'].head()
avg_temp_c min_temp_c max_temp_c precipitation_mm humidity_pct
date
2026-07-01 27.5 21.8 31.0 0.0 62.0
2026-07-02 32.2 29.0 37.0 8.6 48.0
2026-07-03 32.9 26.6 37.1 3.8 60.0
2026-07-04 31.7 28.5 35.6 0.0 50.0
2026-07-05 30.4 26.5 36.0 0.0 70.0
print(weather.loc['2026-07', 'avg_temp_c'].mean().round(1))
print(weather.loc['2026-01', 'avg_temp_c'].mean().round(1))
29.3
7.4

Part 4: Resampling to Monthly Totals#

monthly = weather.resample('ME').agg(
    avg_temp_c=('avg_temp_c', 'mean'),
    total_precip_mm=('precipitation_mm', 'sum'),
    avg_humidity_pct=('humidity_pct', 'mean')
).round(1)
monthly
avg_temp_c total_precip_mm avg_humidity_pct
date
2026-01-31 7.4 36.4 55.8
2026-02-28 11.6 53.4 62.7
2026-03-31 17.0 57.2 61.9
2026-04-30 22.7 49.8 60.6
2026-05-31 27.3 88.6 57.1
2026-06-30 29.7 66.0 65.0
2026-07-31 29.3 54.1 59.2
2026-08-31 24.6 51.6 59.9
2026-09-30 18.9 37.2 61.5
2026-10-31 13.2 38.0 63.0
2026-11-30 8.5 76.4 63.6
2026-12-31 6.5 63.8 62.1

Part 5: Rolling Averages#

weather['rolling_7d_temp'] = weather['avg_temp_c'].rolling(window=7).mean()
weather[['avg_temp_c', 'rolling_7d_temp']].tail()
avg_temp_c rolling_7d_temp
date
2026-12-27 4.8 4.928571
2026-12-28 7.2 4.914286
2026-12-29 4.0 5.414286
2026-12-30 7.3 5.685714
2026-12-31 9.8 6.085714

Flagging Extreme Weather Days#

weather['is_extreme'] = (weather['max_temp_c'] > 28) | (weather['precipitation_mm'] > 15)
print(weather['is_extreme'].sum())
weather[weather['is_extreme']][['max_temp_c', 'precipitation_mm']].head()
131
max_temp_c precipitation_mm
date
2026-03-21 22.3 20.9
2026-03-25 28.1 0.0
2026-03-27 22.8 20.3
2026-04-01 24.9 15.2
2026-04-09 31.3 12.5

Part 6: Seasonal Patterns#

weather_reset = weather.reset_index()
weather_reset['month'] = weather_reset['date'].dt.month_name()
monthly_avg_temp = weather_reset.groupby('month')['avg_temp_c'].mean().round(1)
monthly_avg_temp = monthly_avg_temp.reindex(['January', 'February', 'March', 'April', 'May', 'June', 'July', 'August', 'September', 'October', 'November', 'December'])
monthly_avg_temp
month
January       7.4
February     11.6
March        17.0
April        22.7
May          27.3
June         29.7
July         29.3
August       24.6
September    18.9
October      13.2
November      8.5
December      6.5
Name: avg_temp_c, dtype: float64

Part 7: Visualizing the Results#

import matplotlib
matplotlib.use('Agg')
import matplotlib.pyplot as plt

fig, ax = plt.subplots(figsize=(10, 5))
ax.plot(weather.index, weather['avg_temp_c'], color='gray', alpha=0.4, label='Daily avg temp')
ax.plot(weather.index, weather['rolling_7d_temp'], color='crimson', linewidth=2, label='7-day rolling average')
ax.set_title('Daily Temperature Across the Year')
ax.set_ylabel('Temperature (C)')
ax.legend()
plt.tight_layout()
plt.savefig('temperature_trend.png', dpi=150)
plt.close(fig)
print('Saved temperature_trend.png')
Saved temperature_trend.png
fig, ax = plt.subplots(figsize=(8, 5))
monthly['total_precip_mm'].plot(kind='bar', ax=ax, color='steelblue')
ax.set_title('Total Precipitation by Month')
ax.set_ylabel('Precipitation (mm)')
ax.set_xticklabels([d.strftime('%b') for d in monthly.index], rotation=45)
plt.tight_layout()
plt.savefig('monthly_precipitation.png', dpi=150)
plt.close(fig)
print('Saved monthly_precipitation.png')
Saved monthly_precipitation.png

Wrap-Up: What You Learned#

  • Generating a realistic synthetic year of weather with a seasonal sine wave, then saving and reloading with to_csv and read_csv.
  • Spotting and cleaning missing values with isna and interpolate.
  • Setting a datetime index and slicing it with partial strings like '2026-07'.
  • Resampling to monthly totals and averages with resample plus agg.
  • Rolling averages for smoothing noisy daily data, and boolean flags for extreme days.
  • Seasonal analysis with dt.month_name, groupby, and reindex for correct calendar order.
  • You went from a raw synthetic weather feed to a full seasonal analysis with charts. If you want the next dataset in this series to land in your feed automatically, subscribing is the move see you in the next one.

Found this useful?

All lessons, notebooks and datasets here are free. If they helped you, a coffee keeps new lessons coming.