Mathew K Analytics

Lesson 33 · Python for Data Science

1 Handling Missing Values in Python for Data Cleaning

Today, you will learn how to work with missing values in Python and why it is important. Missing values appear in real-life data all the time. Learning how…

⬇ Download notebookOpen in Colab ↗

What you'll learn

Data

No separate download needed — the notebook creates or downloads everything it uses.

📓 Full notebook

Download .ipynb
 

Welcome to Python: Handling Missing Values#

Today, you will learn how to work with missing values in Python and why it is important.

Missing values appear in real-life data all the time. Learning how to spot and handle them helps you build reliable, useful programs.

Let us get started!

# The special value for missing data in Python is called None
my_value = None
print(my_value)
None

Why does missing data matter?#

If we ignore missing values, our results can be wrong or misleading.

Python gives us tools to check, handle, and work safely with missing values.

# Check if a variable is missing with is operator
value = None
if value is None:
    print('The value is missing.')
else:
    print('The value is present.')
    
The value is missing.
# User input and missing values
user_age = input('Enter your age, or press Enter to skip: ')
if user_age == '':
    user_age = None
print('User age:', user_age)
User age: None
 
# Detecting missing entries in a list
data = [45, None, 35, None, 65]
for item in data:
    if item is None:
        print('Missing entry found!')
    else:
        print('Entry is:', item)
        
Entry is: 45
Missing entry found!
Entry is: 35
Missing entry found!
Entry is: 65
# Counting missing values in a list
scores = [55, None, 67, 89, None, 40]
missing_count = 0
for score in scores:
    if score is None:
        missing_count += 1
print('There are', missing_count, 'missing entries.')
There are 2 missing entries.
# Replacing missing values with a default
temperatures = [22, None, 19, None, 21]
for i in range(len(temperatures)):
    if temperatures[i] is None:
        temperatures[i] = 20
print('Updated temperatures:', temperatures)
Updated temperatures: [22, 20, 19, 20, 21]
# Remove missing values from a list
data_with_gaps = [3, None, 8, None, 2]
cleaned = []
for item in data_with_gaps:
    if item is not None:
        cleaned.append(item)
print('Clean data:', cleaned)
Clean data: [3, 8, 2]
# Find the first non-missing value
answers = [None, None, 5, 7]
for answer in answers:
    if answer is not None:
        print('First answer is:', answer)
        break
    
First answer is: 5

Missing values in dictionaries#

Dictionaries can also have missing or unknown values. We can check for them using None, just like in lists.

# Example: Handling missing values in dictionary
info = {'name': 'Sam', 'birthday': None, 'score': 88}
if info['birthday'] is None:
    print('Birthday is missing!')
    
Birthday is missing!
# Safe dictionary lookup with get()
contacts = {'Dana': '555-1234'}
phone = contacts.get('Alex', None)
if phone is None:
    print('Phone number for Alex is missing.')
    
Phone number for Alex is missing.
# Handling missing data in real-world text
text = input('Enter your city, or press Enter if missing: ')
if text == '':
    city = None
else:
    city = text
print('City is:', city)
City is: Toronto
 
# Calculate average, ignoring missing numbers
grades = [90, None, 85, None, 78]
total = 0
count = 0
for grade in grades:
    if grade is not None:
        total += grade
        count += 1
if count > 0:
    average = total / count
    print('Average grade:', average)
else:
    print('No grades to average.')
    
Average grade: 84.33333333333333
# List comprehensions to filter missing data
data = [10, None, 30, None, 50]
cleaned = [x for x in data if x is not None]
print('Values without missing data:', cleaned)
Values without missing data: [10, 30, 50]
# Sorting data, putting missing values at the end
results = [25, None, 15, 40, None, 30]
sorted_results = sorted(results, key=lambda x: (x is None, x))
print('Sorted data:', sorted_results)
Sorted data: [15, 25, 30, 40, None, None]
# Filtering out outliers and missing data
numbers = [100, None, 150, 20, 300, None, 40]
filtered = [x for x in numbers if x is not None and x < 200]
print('Filtered data:', filtered)
Filtered data: [100, 150, 20, 40]

Mini-project: Clean and summarize survey data#

Imagine you collected favorite color responses from friends, but some are missing.

Let us clean the data and count how many people answered.

# Mini-project, part 1: Clean favorite colors
responses = ['blue', None, 'green', '', 'red', None, '', 'blue']
cleaned = []
for color in responses:
    if color and color is not None:
        cleaned.append(color)
print('Clean responses:', cleaned)
Clean responses: ['blue', 'green', 'red', 'blue']
# Mini-project, part 2: Count answers
color_count = {}
for color in cleaned:
    if color not in color_count:
        color_count[color] = 1
    else:
        color_count[color] += 1
print('Answer counts:', color_count)
Answer counts: {'blue': 2, 'green': 1, 'red': 1}
# Best practice: Document missing values with comments
data = [12, None, 18]
# None means 'value not recorded' for this data.
print(data)
[12, None, 18]
# What if you forget to check for missing data?
values = [5, None, 10]
try:
    total = 0
    for v in values:
        total += v
    print('Total:', total)
except TypeError:
    print('Error: Tried to add a missing value!')
    
Error: Tried to add a missing value!
# Quick tip: Use or to set a default if missing
result = None
output = result or 'Unknown'
print('Result is:', output)
Result is: Unknown

Challenge: Fix the broken program#

Suppose you have a list called answers, but do not know which are missing or blanks.

Write code to create a new list with only real answers.

Lesson recap#

You now know how to spot, check, and handle missing values in Python.

Remember to always check for None before using values in math or logic. Missing values do not have to break your code. Handle them safely and keep your data solid!

Thank you for watching!#

If you enjoyed this lesson, give a thumbs up, comment your progress, subscribe for more, and share with your friends.

See you next time on our Python learning series!

Found this useful?

All lessons, notebooks and datasets here are free. If they helped you, a coffee keeps new lessons coming.