Handling missing values in Series in Data Analysis Python - Mini Project: Build & Apply

Handling missing values in Series

📖 Scenario: Imagine you have collected daily temperature readings for a week, but some days the sensor failed and the data is missing. You want to clean this data to analyze it properly.

🎯 Goal: You will create a pandas Series with temperature data including missing values, set a threshold for acceptable missing data, fill missing values with the average temperature, and finally display the cleaned data.

📋 What You'll Learn

Create a pandas Series named temps with the exact values: 22.5, None, 21.0, None, 23.5, 24.0, None

Create a variable named max_missing and set it to 2

Fill missing values in temps with the mean of the available temperatures and assign it to cleaned_temps

Print the cleaned_temps Series

💡 Why This Matters

🌍 Real World

Handling missing data is common in real-world datasets like weather, sales, or health records. Cleaning data helps make better decisions.

💼 Career

Data scientists and analysts often clean and prepare data before analysis. Knowing how to handle missing values is a key skill.

1

Create the temperature data Series

Create a pandas Series called temps with these exact values in order: 22.5, None, 21.0, None, 23.5, 24.0, None

Data Analysis Python

import pandas as pd
# Create the Series temps with the given values
# Your code here

Hint

Use pd.Series and pass a list with the exact values including None for missing data.

2

Set the maximum allowed missing values

Create a variable called max_missing and set it to 2

Data Analysis Python

import pandas as pd

temps = pd.Series([22.5, None, 21.0, None, 23.5, 24.0, None])
# Set max_missing to 2
# Your code here

Hint

Just assign the number 2 to a variable named max_missing.

3

Fill missing values with the average temperature

Create a new Series called cleaned_temps by filling the missing values in temps with the mean of the available temperatures

Data Analysis Python

import pandas as pd

temps = pd.Series([22.5, None, 21.0, None, 23.5, 24.0, None])

max_missing = 2
# Fill missing values in temps with the mean and assign to cleaned_temps
# Your code here

Hint

Use temps.fillna() and pass temps.mean() as the value to fill missing data.

4

Display the cleaned temperature data

Print the cleaned_temps Series to show the temperature data with missing values filled

Data Analysis Python

import pandas as pd

temps = pd.Series([22.5, None, 21.0, None, 23.5, 24.0, None])

max_missing = 2

cleaned_temps = temps.fillna(temps.mean())
# Print cleaned_temps
# Your code here

Hint

Use print(cleaned_temps) to display the Series.

Practice

1. What does the isna() method do when applied to a pandas Series?

easy

A. It replaces missing values with zeros.

B. It removes all missing values from the Series.

C. It counts the number of missing values in the Series.

D. It returns a Series of booleans indicating which values are missing (NaN).

Solution

1. Step 1: Understand the purpose of isna()

   The isna() method checks each element in the Series and returns True if the value is missing (NaN), otherwise False.

2. Step 2: Compare with other options

   Removing values is done by dropna(), replacing by fillna(), and counting missing values requires sum() on the boolean Series.

3. Final Answer:

   It returns a Series of booleans indicating which values are missing (NaN). -> Option D

4. Quick Check:

   isna() = boolean mask [OK]

Hint: Remember: isna() marks missing values as True [OK]

Common Mistakes:

• Confusing isna() with dropna()
• Thinking isna() replaces values
• Assuming isna() counts missing values

2. Which of the following is the correct syntax to remove missing values from a pandas Series named data?

easy

A. data.dropna()

B. data.isna()

C. data.fillna()

D. data.remove_na()

Solution

1. Step 1: Identify the method to remove missing values

   The method dropna() removes missing values from a Series.

2. Step 2: Check other options for correctness

   isna() finds missing values, fillna() replaces them, and remove_na() is not a valid pandas method.

3. Final Answer:

   data.dropna() -> Option A

4. Quick Check:

   dropna() removes NaNs [OK]

Hint: Use dropna() to delete missing values from Series [OK]

Common Mistakes:

• Using fillna() instead of dropna() to remove values
• Calling a non-existent method remove_na()
• Confusing isna() with dropna()

3. Given the Series import pandas as pd s = pd.Series([1, None, 3, None, 5]) result = s.fillna(0), what is the value of result?

medium

A. [1, 0, 3, 0, 5]

B. [1, None, 3, None, 5]

C. [0, 0, 0, 0, 0]

D. [1, 1, 3, 3, 5]

Solution

1. Step 1: Understand fillna(0) on the Series

   The fillna(0) replaces all missing values (None/NaN) with 0.

2. Step 2: Apply fillna(0) to the Series

   Original Series: [1, None, 3, None, 5]. After fillna(0): [1, 0, 3, 0, 5].

3. Final Answer:

   [1, 0, 3, 0, 5] -> Option A

4. Quick Check:

   fillna(0) replaces NaN with 0 [OK]

Hint: fillna(value) replaces NaNs with value [OK]

Common Mistakes:

• Expecting fillna() to remove values
• Thinking fillna(0) replaces all values with 0
• Confusing None with zero already present

4. What is wrong with this code snippet to remove missing values from Series s?

clean_s = s.dropna(inplace=True)
print(clean_s)

medium

A. dropna() does not have an inplace parameter.

B. print(clean_s) will print the original Series with NaNs.

C. dropna(inplace=True) returns None, so clean_s is None.

D. dropna(inplace=True) modifies a copy, not the original Series.

Solution

1. Step 1: Understand dropna(inplace=True) behavior

   When inplace=True is used, dropna() modifies the Series in place and returns None.

2. Step 2: Analyze assignment and print

   Assigning clean_s = s.dropna(inplace=True) sets clean_s to None. Printing it shows None, not the cleaned Series.

3. Final Answer:

   dropna(inplace=True) returns None, so clean_s is None. -> Option C

4. Quick Check:

   inplace=True returns None [OK]

Hint: dropna(inplace=True) returns None, don't assign it [OK]

Common Mistakes:

• Assigning dropna(inplace=True) to a variable
• Expecting dropna() to return cleaned Series with inplace=True
• Confusing inplace behavior with default dropna()

5. You have a Series s = pd.Series([1, None, 2, None, 100]). You want to replace missing values with the mean of the non-missing values. Which code correctly does this?

hard

A. s.replace(None, s.mean())

B. s.fillna(s.mean())

C. s.dropna().mean()

D. s.fillna(s.median())

Solution

1. Step 1: Calculate mean of non-missing values

   Using s.mean() calculates the average ignoring missing values.

2. Step 2: Replace missing values with mean

   fillna(s.mean()) replaces NaNs with the calculated mean.

3. Step 3: Check other options

   fillna(s.median()) replaces with median, not mean. dropna().mean() calculates mean but does not fill. replace(None, s.mean()) does not work properly for NaN.

4. Final Answer:

   s.fillna(s.mean()) -> Option B

5. Quick Check:

   fillna(mean) replaces NaNs with average [OK]

Hint: Use fillna(s.mean()) to replace NaNs with average [OK]

Common Mistakes:

• Using median instead of mean
• Calculating mean but not filling missing values
• Using replace() which doesn't handle NaN properly