Filling missing values (fillna) in Data Analysis Python - Mini Project: Build & Apply

Filling Missing Values (fillna)

📖 Scenario: Imagine you work for a small online store. You have a table of sales data, but some sales amounts are missing. You want to fill these missing values with zero so you can calculate total sales without errors.

🎯 Goal: You will create a small sales data table with missing values, set a fill value, fill the missing values using fillna, and then print the cleaned data.

📋 What You'll Learn

Create a pandas DataFrame with missing values

Create a variable to hold the fill value

Use the fillna method to fill missing values

Print the cleaned DataFrame

💡 Why This Matters

🌍 Real World

Cleaning missing data is a common task in data analysis to prepare data for calculations and visualizations.

💼 Career

Data analysts and scientists often need to handle missing data to ensure accurate reports and models.

1

Create sales data with missing values

Create a pandas DataFrame called sales_data with one column named 'sales' and these values: 100, None, 250, None, 400.

Data Analysis Python

import pandas as pd
# Create the sales_data DataFrame with missing values
# Your code here

Hint

Use pd.DataFrame with a dictionary where the key is 'sales' and the value is the list of numbers including None for missing values.

2

Set the fill value

Create a variable called fill_value and set it to 0 to use for filling missing sales.

Data Analysis Python

import pandas as pd
sales_data = pd.DataFrame({'sales': [100, None, 250, None, 400]})
# Create fill_value variable and set it to 0
# Your code here

Hint

Just create a variable named fill_value and assign 0 to it.

3

Fill missing values using fillna

Create a new DataFrame called cleaned_sales by filling missing values in sales_data using the fillna method with the variable fill_value.

Data Analysis Python

import pandas as pd
sales_data = pd.DataFrame({'sales': [100, None, 250, None, 400]})
fill_value = 0
# Use fillna to create cleaned_sales DataFrame
# Your code here

Hint

Use sales_data.fillna(fill_value) and assign it to cleaned_sales.

4

Print the cleaned sales data

Print the cleaned_sales DataFrame to see the missing values replaced by zero.

Data Analysis Python

import pandas as pd
sales_data = pd.DataFrame({'sales': [100, None, 250, None, 400]})
fill_value = 0
cleaned_sales = sales_data.fillna(fill_value)
# Print cleaned_sales DataFrame
# Your code here

Hint

Use print(cleaned_sales) to display the DataFrame.

Practice

1. What does the fillna() function do in a pandas DataFrame?

easy

A. It converts all values to strings.

B. It replaces missing values with a specified value or method.

C. It sorts the DataFrame by missing values.

D. It deletes rows with missing values.

Solution

1. Step 1: Understand the purpose of fillna()

   The fillna() function is designed to handle missing data by replacing NaN values.

2. Step 2: Identify what fillna() replaces missing values with

   It can replace missing values with a fixed value or use methods like forward fill or backward fill.

3. Final Answer:

   It replaces missing values with a specified value or method. -> Option B

4. Quick Check:

   fillna() replaces NaN values = D [OK]

Hint: fillna() fills missing spots, not deletes or sorts [OK]

Common Mistakes:

• Confusing fillna() with dropna() which deletes rows
• Thinking fillna() sorts data
• Assuming fillna() changes data types

2. Which of the following is the correct syntax to fill missing values in column 'Age' with 30 in a pandas DataFrame df?

easy

A. df['Age'].fill(30)

B. df.fillna['Age'](30)

C. df.fillna('Age', 30)

D. df['Age'].fillna(30)

Solution

1. Step 1: Identify the correct method call on a DataFrame column

   To fill missing values in a specific column, call fillna() on that column like df['Age'].fillna(30).

2. Step 2: Check syntax correctness

   df['Age'].fillna(30) uses correct syntax: column selection with brackets and method call with parentheses and argument.

3. Final Answer:

   df['Age'].fillna(30) -> Option D

4. Quick Check:

   Correct method call syntax = A [OK]

Hint: Use brackets for column, parentheses for fillna() [OK]

Common Mistakes:

• Using square brackets instead of parentheses for fillna()
• Calling fillna on DataFrame with wrong syntax
• Using fill() instead of fillna()

3. Given the DataFrame df below, what will be the output of df['Score'].fillna(0)?

import pandas as pd
df = pd.DataFrame({'Score': [10, None, 20, None, 30]})
print(df['Score'].fillna(0))

medium

A. [0, 0, 0, 0, 0]

B. [10, None, 20, None, 30]

C. [10, 0, 20, 0, 30]

D. [10, 10, 20, 20, 30]

Solution

1. Step 1: Understand fillna(0) effect on NaN values

   Calling fillna(0) replaces all missing (NaN) values with 0.

2. Step 2: Apply fillna(0) to the 'Score' column

   The original column has values [10, NaN, 20, NaN, 30]. After fillna(0), NaNs become 0, so result is [10, 0, 20, 0, 30].

3. Final Answer:

   [10, 0, 20, 0, 30] -> Option C

4. Quick Check:

   NaN replaced by 0 = C [OK]

Hint: fillna(value) replaces NaN with value directly [OK]

Common Mistakes:

• Expecting fillna() to remove rows
• Thinking fillna() copies previous values by default
• Confusing fillna(0) with fillna(method='ffill')

4. What is wrong with this code snippet to fill missing values in DataFrame df?

df.fillna(method='forward')

medium

A. The method name should be 'ffill', not 'forward'.

B. fillna() cannot use methods to fill missing values.

C. The method argument should be a number, not a string.

D. fillna() must be called on a column, not the whole DataFrame.

Solution

1. Step 1: Check valid method names for fillna()

   fillna() supports method='ffill' for forward fill and method='bfill' for backward fill.

2. Step 2: Identify the incorrect method name

   The code uses method='forward', which is invalid. The correct name is 'ffill'.

3. Final Answer:

   The method name should be 'ffill', not 'forward'. -> Option A

4. Quick Check:

   Correct method name = 'ffill' [OK]

Hint: Use 'ffill' or 'bfill' for method in fillna() [OK]

Common Mistakes:

• Using 'forward' instead of 'ffill'
• Thinking fillna() can't use methods
• Calling fillna() only on columns, not DataFrame

5. You have a DataFrame df with missing values in multiple columns. You want to fill missing values in column 'A' with the mean of 'A', and in column 'B' with the previous valid value. Which code correctly does this?

hard

A. df['A'] = df['A'].fillna(df['A'].mean())
df['B'] = df['B'].fillna(method='ffill')

B. df.fillna({'A': df['A'].mean(), 'B': 'ffill'})

C. df['A'].fillna(df['A'].mean())
df['B'].fillna(method='bfill')

D. df.fillna(method={'A': 'mean', 'B': 'ffill'})

Solution

1. Step 1: Fill column 'A' with its mean

   Calculate mean with df['A'].mean() and fill missing values using fillna() on column 'A'. Assign back to df['A'].

2. Step 2: Fill column 'B' with previous valid value

   Use fillna(method='ffill') on column 'B' to forward fill missing values. Assign back to df['B'].

3. Final Answer:

   df['A'] = df['A'].fillna(df['A'].mean()) df['B'] = df['B'].fillna(method='ffill') -> Option A

4. Quick Check:

   Mean fill + forward fill = B [OK]

Hint: Fill columns separately with correct methods, assign back [OK]

Common Mistakes:

• Using fillna() without assignment, so no change saved
• Passing 'ffill' as string value instead of method argument
• Trying to use method='mean' which is invalid