Filling missing values with fillna() in Pandas - Step-by-Step Execution

Concept Flow - Filling missing values with fillna()

Start with DataFrame

↓

Identify missing values

↓

Choose fill value or method

↓

Apply fillna()

↓

Get DataFrame with no missing values

↓

End

Start with a DataFrame that has missing values, decide how to fill them, apply fillna(), and get a DataFrame without missing values.

Execution Sample

Pandas

import pandas as pd

df = pd.DataFrame({'A': [1, None, 3], 'B': [None, 2, 3]})
df_filled = df.fillna(0)
print(df_filled)

This code creates a DataFrame with missing values and fills them with 0 using fillna().

Execution Table

Step	DataFrame State	fillna() Argument	Action	Resulting DataFrame
1	{'A': [1, None, 3], 'B': [None, 2, 3]}	0	Identify missing values in columns A and B	{'A': [1, None, 3], 'B': [None, 2, 3]} (missing values present)
2	Same as above	0	Apply fillna(0) to replace all None with 0	{'A': [1, 0, 3], 'B': [0, 2, 3]} (missing values replaced)
3	Filled DataFrame	N/A	Print the filled DataFrame	A B 0 1.0 0.0 1 0.0 2.0 2 3.0 3.0
4	Filled DataFrame	N/A	End of execution	No missing values remain

💡 All missing values replaced by 0, no None left in DataFrame

Variable Tracker

Variable	Start	After fillna(0)	Final
df	{'A': [1, None, 3], 'B': [None, 2, 3]}	{'A': [1, None, 3], 'B': [None, 2, 3]}	Unchanged
df_filled	Not defined	{'A': [1, 0, 3], 'B': [0, 2, 3]}	Filled DataFrame with no missing values

Key Moments - 3 Insights

Why does df not change after fillna(0)?

Because fillna() returns a new DataFrame by default and does not modify df in place. See execution_table step 2 where df remains unchanged.

What happens if we use fillna() without an argument?

fillna() requires a value or method to fill missing data. Without an argument, it will raise an error. This is why we specify 0 in step 2.

Can fillna() fill missing values differently for each column?

Yes, by passing a dictionary with column names as keys and fill values as values. This is not shown here but is a common use.

Visual Quiz - 3 Questions

Look at the execution table at step 2. What value replaces the missing values?

A 1

B 0

C None

D NaN

💡 Hint

Check the 'fillna() Argument' and 'Resulting DataFrame' columns at step 2.

At which step does the DataFrame have no missing values?

A Step 2

B Step 1

C Step 3

D Step 4

💡 Hint

Look at the 'Resulting DataFrame' column to see when missing values are replaced.

If we want to change df directly without creating df_filled, what should we do?

A Use df.fillna(0) without assignment

B Use df.fillna() without arguments

C Use df.fillna(0, inplace=True)

D Use df.fillna(0).copy()

💡 Hint

Recall that fillna() returns a new DataFrame unless inplace=True is used.

Concept Snapshot

fillna() replaces missing values in a DataFrame.
Syntax: df.fillna(value, inplace=False)
By default, returns a new DataFrame.
Use inplace=True to modify original.
Value can be a scalar or dict per column.
Missing values are replaced by the given value.

Full Transcript

We start with a DataFrame that has missing values represented by None. We want to fill these missing spots with a value, here zero. Using fillna(0), we create a new DataFrame where all None values are replaced by 0. The original DataFrame remains unchanged unless we use inplace=True. This method helps clean data by removing missing values in a simple way.

Practice

1. What does the fillna() function do in pandas?

easy

A. It changes the data type of columns.

B. It removes all rows with missing values.

C. It sorts the DataFrame by column values.

D. It replaces missing values with specified values.

Solution

1. Step 1: Understand the purpose of fillna()

   The fillna() function is designed to handle missing data by replacing NaN values with a specified value or method.

2. Step 2: Compare options with fillna() behavior

   Options B, C, and D describe other DataFrame operations, but only It replaces missing values with specified values. correctly describes filling missing values.

3. Final Answer:

   It replaces missing values with specified values. -> Option D

4. Quick Check:

   fillna() = replace missing values [OK]

Hint: fillna() fills missing spots with your chosen value [OK]

Common Mistakes:

• Confusing fillna() with dropna() which removes rows
• Thinking fillna() sorts data
• Assuming fillna() changes data types

2. Which of the following is the correct syntax to fill missing values in a DataFrame df with zero using fillna()?

easy

A. df.fillna(na=0)

B. df.fillna = 0

C. df.fillna(0)

D. df.fillna(0, value=0)

Solution

1. Step 1: Recall fillna() method syntax

   The correct way to fill missing values is calling fillna() with the value as the first argument, like df.fillna(0).

2. Step 2: Evaluate each option

   df.fillna = 0 tries to assign to fillna which is incorrect. df.fillna(na=0) uses a wrong keyword argument. df.fillna(0, value=0) passes 0 both positionally (as value) and explicitly as value=0 keyword, causing TypeError: multiple values for argument 'value'. Only C is correct.

3. Final Answer:

   df.fillna(0) -> Option C

4. Quick Check:

   fillna(value) fills missing with value [OK]

Hint: Use fillna(value) to fill missing values simply [OK]

Common Mistakes:

• Assigning fillna instead of calling it as a method
• Using wrong argument names like na=0
• Confusing inplace parameter usage

3. Given the DataFrame:

import pandas as pd
df = pd.DataFrame({'A': [1, None, 3], 'B': [None, 2, None]})
filled = df.fillna({'A': 0, 'B': 5})
print(filled)

What is the output?

medium

A. A B
0 1 5
1 0 2
2 3 5

B. A B
0 1 NaN
1 0 2
2 3 NaN

C. A B
0 1 0
1 0 2
2 3 0

D. A B
0 0 5
1 0 2
2 0 5

Solution

1. Step 1: Understand fillna() with dict argument

   Using a dictionary with fillna() fills missing values in each column with the corresponding value from the dict. Here, column 'A' missing values become 0, and column 'B' missing values become 5.

2. Step 2: Apply fillna() to each column

   Original df has NaN at df.loc[1, 'A'] and df.loc[0, 'B'], df.loc[2, 'B']. After fillna, these become 0 for 'A' and 5 for 'B'. The rest of the values remain unchanged.

3. Final Answer:

   A B 0 1 5 1 0 2 2 3 5 -> Option A

4. Quick Check:

   fillna(dict) fills columns separately [OK]

Hint: Use dict in fillna() to fill columns differently [OK]

Common Mistakes:

• Expecting fillna() to fill all columns with same value
• Confusing NaN replacement with zero by default
• Misreading which values are replaced

4. What is wrong with this code snippet?

import pandas as pd
df = pd.DataFrame({'X': [None, 2, None]})
df.fillna(inplace=True, 0)
print(df)

medium

A. fillna() keyword arguments are incorrectly used.

B. The arguments to fillna() are in wrong order.

C. The value argument should come before inplace.

D. fillna() does not have an inplace parameter.

Solution

1. Step 1: Check fillna() parameter names and order

   In Python functions, all positional arguments must come before any keyword arguments.

2. Step 2: Identify incorrect argument order

   The code uses inplace=True (keyword) followed by 0 (positional for value), which raises SyntaxError: positional argument follows keyword argument. Correct is df.fillna(0, inplace=True).

3. Final Answer:

   The arguments to fillna() are in wrong order. -> Option B

4. Quick Check:

   Positional args before keywords [OK]

Hint: Pass value first, then inplace as keyword in fillna() [OK]

Common Mistakes:

• Swapping argument order without keywords
• Using inplace without value argument
• Confusing positional and keyword arguments

5. You have a DataFrame df with columns 'Score' and 'Grade'. 'Score' has missing values. You want to fill missing 'Score' values with the mean of 'Score' and then fill missing 'Grade' values with 'Unknown'. Which code correctly does this?

hard

A. df.fillna({'Score': df['Score'].mean(), 'Grade': 'Unknown'}, inplace=True)

B. df['Score'].fillna(df['Score'].mean())
df['Grade'].fillna('Unknown')

C. df.fillna({'Score': 'Unknown', 'Grade': df['Grade'].mean()}, inplace=True)

D. df.fillna(df.mean(), inplace=True)
df['Grade'].fillna('Unknown')

Solution

1. Step 1: Calculate mean and prepare fill dictionary

   We calculate the mean of 'Score' using df['Score'].mean() and want to fill missing 'Score' with this mean. For 'Grade', missing values should be filled with 'Unknown'.

2. Step 2: Use fillna() with a dictionary and inplace=True

   Using df.fillna({'Score': mean_value, 'Grade': 'Unknown'}, inplace=True) fills missing values in both columns correctly in one step.

3. Step 3: Evaluate other options

   df['Score'].fillna(df['Score'].mean()) df['Grade'].fillna('Unknown') does not assign back the filled columns, so missing values remain. df.fillna({'Score': 'Unknown', 'Grade': df['Grade'].mean()}, inplace=True) swaps fill values incorrectly. df.fillna(df.mean(), inplace=True) df['Grade'].fillna('Unknown') fills 'Score' with its column mean (assuming 'Score' numeric) but the second line returns a new Series without assignment or inplace=True, leaving 'Grade' unchanged.

4. Final Answer:

   df.fillna({'Score': df['Score'].mean(), 'Grade': 'Unknown'}, inplace=True) -> Option A

5. Quick Check:

   Use dict with fillna() and inplace=True for multiple columns [OK]

Hint: Fill multiple columns at once with dict and inplace=True [OK]

Common Mistakes:

• Not assigning back after fillna() without inplace
• Mixing fill values between columns
• Using fillna() separately without inplace or assignment