Mathew K Analytics

Lesson 35 · Mastering Pandas

Converting Strings to Datetime and Extracting Date Components Using Pandas in Python

In this lesson, we will learn how to work with dates and times in pandas. You will see how to convert string columns to real datetime types. We will also…

⬇ Download notebookOpen in Colab ↗

What you'll learn

Data

No separate download needed — the notebook creates or downloads everything it uses.

📓 Full notebook

Download .ipynb

Converting Strings to Datetime and Extracting Components in Pandas#

In this lesson, we will learn how to work with dates and times in pandas.

You will see how to convert string columns to real datetime types.

We will also learn how to extract useful parts, like the year, month, or weekday, for analysis.

Handling dates is an essential skill in data analysis, finance, and much more.

import warnings
warnings.filterwarnings("ignore")

# Let us set up the Flights dataset, which is great for learning time handling.
import seaborn as sns
import numpy as np
import pandas as pd
np.random.seed(42)
df = sns.load_dataset('flights')
print(df.shape)
print(df.head(3))
(144, 3)
   year month  passengers
0  1949   Jan         112
1  1949   Feb         118
2  1949   Mar         132

Why are datetime types useful?#

Most real datasets have dates as strings.

Dates as true pandas datetime types let you sort, filter, plot, and extract features easily.

Let us look at the columns of our dataset.

print(df.columns)
print(df.dtypes)
Index(['year', 'month', 'passengers'], dtype='object')
year             int64
month         category
passengers       int64
dtype: object
# Creating a single column with both year and month as a string.
df['date_str'] = df['year'].astype(str) + '-' + df['month'].astype(str)
print(df[['year', 'month', 'date_str']].head())
   year month  date_str
0  1949   Jan  1949-Jan
1  1949   Feb  1949-Feb
2  1949   Mar  1949-Mar
3  1949   Apr  1949-Apr
4  1949   May  1949-May
# Try to sort by date_str.
print(df.sort_values('date_str').head(3))
    year month  passengers  date_str
3   1949   Apr         129  1949-Apr
7   1949   Aug         148  1949-Aug
11  1949   Dec         118  1949-Dec

Converting date strings to real datetime#

Pandas can turn many kinds of date strings into datetime type with the to_datetime function.

This will allow powerful date operations.

df['date'] = pd.to_datetime(df['date_str'], format='%Y-%b')
print(df[['date_str', 'date']].head())
   date_str       date
0  1949-Jan 1949-01-01
1  1949-Feb 1949-02-01
2  1949-Mar 1949-03-01
3  1949-Apr 1949-04-01
4  1949-May 1949-05-01
# Check the type of the new column.
print(type(df.loc[0, 'date']))
<class 'pandas._libs.tslibs.timestamps.Timestamp'>

Date features: extracting year, month, and more#

Once you have datetimes, you can easily get features like year, month, quarter, weekday, day, and more.

These are just attributes of the pandas datetime column.

df['year_extracted'] = df['date'].dt.year
df['month_number'] = df['date'].dt.month
df['weekday'] = df['date'].dt.weekday
print(df[['date', 'year_extracted', 'month_number', 'weekday']].head())
        date  year_extracted  month_number  weekday
0 1949-01-01            1949             1        5
1 1949-02-01            1949             2        1
2 1949-03-01            1949             3        1
3 1949-04-01            1949             4        4
4 1949-05-01            1949             5        6
# You can get the weekday as the name, too.
df['weekday_name'] = df['date'].dt.day_name()
print(df[['date', 'weekday_name']].head())
        date weekday_name
0 1949-01-01     Saturday
1 1949-02-01      Tuesday
2 1949-03-01      Tuesday
3 1949-04-01       Friday
4 1949-05-01       Sunday
# Challenge: What date had the most passengers?
max_idx = df['passengers'].idxmax()
print(df.loc[max_idx, ['date', 'passengers']])
date          1960-07-01 00:00:00
passengers                    622
Name: 138, dtype: object
# Example: Count total passengers by weekday name.
weekday_totals = df.groupby('weekday_name')['passengers'].sum().sort_values(ascending=False)
print(weekday_totals)
weekday_name
Friday       6231
Tuesday      6213
Sunday       5891
Thursday     5711
Monday       5528
Wednesday    5423
Saturday     5366
Name: passengers, dtype: int64
# You can also filter the DataFrame for weekends.
weekends = df[df['weekday_name'].isin(['Saturday', 'Sunday'])]
print(weekends.head(3))
   year month  passengers  date_str       date  year_extracted  month_number  \
0  1949   Jan         112  1949-Jan 1949-01-01            1949             1   
4  1949   May         121  1949-May 1949-05-01            1949             5   
9  1949   Oct         119  1949-Oct 1949-10-01            1949            10   

   weekday weekday_name  
0        5     Saturday  
4        6       Sunday  
9        5     Saturday  
# Visualize passengers over time using the datetime index.
import matplotlib.pyplot as plt
plt.figure(figsize=(10, 4))
plt.plot(df['date'], df['passengers'], marker='o')
plt.title('Monthly Airline Passengers Over Time')
plt.xlabel('Date')
plt.ylabel('Passengers')
plt.grid(True)
plt.tight_layout()
plt.show()
No description has been provided for this image

Real-World Tip#

Always convert date columns to pandas datetime as soon as you load new data.

That way, you avoid sorting bugs and can easily use date filtering or extraction features.

# Try: Select all months in 1952.
in_1952 = df[df['date'].dt.year == 1952]
print(in_1952.head())
    year month  passengers  date_str       date  year_extracted  month_number  \
36  1952   Jan         171  1952-Jan 1952-01-01            1952             1   
37  1952   Feb         180  1952-Feb 1952-02-01            1952             2   
38  1952   Mar         193  1952-Mar 1952-03-01            1952             3   
39  1952   Apr         181  1952-Apr 1952-04-01            1952             4   
40  1952   May         183  1952-May 1952-05-01            1952             5   

    weekday weekday_name  
36        1      Tuesday  
37        4       Friday  
38        5     Saturday  
39        1      Tuesday  
40        3     Thursday  
# Next: What is the average number of passengers in July flights?
july_avg = df[df['month'] == 'July']['passengers'].mean()
print(f"Average passengers in July: {july_avg:.2f}")
Average passengers in July: nan
# Sometimes real datasets have multiple datetime columns or tricky formats.
example_dates = pd.Series(['2022/03/01', '01-04-2023', '2023.05.01'])
parsed = pd.to_datetime(example_dates, dayfirst=False, errors='coerce')
print(pd.DataFrame({'original': example_dates, 'parsed': parsed}))
     original     parsed
0  2022/03/01 2022-03-01
1  01-04-2023        NaT
2  2023.05.01        NaT
# Slicing and resampling: What was the total number of passengers in 1950?
total_1950 = df[df['date'].dt.year == 1950]['passengers'].sum()
print(f"Total passengers in 1950: {total_1950}")
Total passengers in 1950: 1676
# Using input to enter your own date string to convert. Try entering '2024-05-15' (YYYY-MM-DD).
user_date_str = input("Type a date (YYYY-MM-DD): ")
user_date = pd.to_datetime(user_date_str)
print("You entered:", user_date)
print("Year:", user_date.year, "Month:", user_date.month, "Day:", user_date.day)
You entered: 2024-05-15 00:00:00
Year: 2024 Month: 5 Day: 15

Review and Next Steps#

Today you learned how to convert strings to datetime and extract date features in pandas.

You practiced with the Flights dataset and saw common real-world tasks for dates.

Next, try using these skills with a new datasetperhaps Titanic or your own CSV with a date column.

Remember: Always check your date columns and convert them early!

If you liked this lesson, like and subscribe to our channel for more tutorials, tips, and projects.

Let us know how you use pandas dates in the comments below!

Happy coding!

Found this useful?

All lessons, notebooks and datasets here are free. If they helped you, a coffee keeps new lessons coming.