Counting missing values in Pandas - Step-by-Step Execution

Concept Flow - Counting missing values

Start with DataFrame

↓

Check each cell for missing?

↓

Count True (missing) per column

↓

Return count per column

↓

Optionally count total missing

↓

End

We start with a DataFrame, check each cell for missing values, count how many missing per column, and optionally get total missing count.

Execution Sample

Pandas

import pandas as pd

df = pd.DataFrame({
  'A': [1, None, 3],
  'B': [None, None, 6]
})
missing_count = df.isna().sum()

This code creates a DataFrame with missing values and counts missing values per column.

Execution Table

Step	Action	DataFrame State	isna() Result	sum() Result
1	Create DataFrame	{'A':[1, None, 3], 'B':[None, None, 6]}	N/A	N/A
2	Check missing with isna()	{'A':[1, None, 3], 'B':[None, None, 6]}	{'A':[False, True, False], 'B':[True, True, False]}	N/A
3	Sum missing per column	{'A':[1, None, 3], 'B':[None, None, 6]}	{'A':[False, True, False], 'B':[True, True, False]}	{'A':1, 'B':2}
4	End	Same as step 1	Same as step 2	Final missing counts per column

💡 All rows processed, missing values counted per column.

Variable Tracker

Variable	Start	After Step 1	After Step 2	After Step 3	Final
df	None	{'A':[1, None, 3], 'B':[None, None, 6]}	{'A':[1, None, 3], 'B':[None, None, 6]}	{'A':[1, None, 3], 'B':[None, None, 6]}	{'A':[1, None, 3], 'B':[None, None, 6]}
missing_mask	None	None	{'A':[False, True, False], 'B':[True, True, False]}	{'A':[False, True, False], 'B':[True, True, False]}	{'A':[False, True, False], 'B':[True, True, False]}
missing_count	None	None	None	{'A':1, 'B':2}	{'A':1, 'B':2}

Key Moments - 3 Insights

Why does df.isna() return a DataFrame of True/False instead of numbers?

df.isna() checks each cell and returns True if missing, False otherwise, as shown in execution_table step 2. This boolean mask helps count missing values.

How does sum() count missing values when applied to booleans?

In Python, True counts as 1 and False as 0. So sum() adds up True values per column, giving the count of missing values (execution_table step 3).

Can we count missing values for the whole DataFrame at once?

Yes, by adding .sum().sum() after isna(), you get total missing values in the entire DataFrame, but here we count per column first (not shown in this trace).

Visual Quiz - 3 Questions

Look at the execution_table at step 3, what is the missing count for column 'B'?

A 1

B 2

C 3

D 0

💡 Hint

Check the sum() Result column in execution_table row with Step 3.

According to variable_tracker, what is the value of missing_mask after step 2?

A {'A':[False, True, False], 'B':[True, True, False]}

B {'A':[True, True, False], 'B':[False, True, False]}

C {'A':[False, False, False], 'B':[True, True, True]}

D None

💡 Hint

Look at the missing_mask row under After Step 2 column in variable_tracker.

If the DataFrame had no missing values, what would be the sum() result?

A {'A':3, 'B':3}

B {'A':1, 'B':1}

C {'A':0, 'B':0}

D None

💡 Hint

Recall that sum() counts True values; no missing means all False, so sum is zero.

Concept Snapshot

Counting missing values in pandas:
- Use df.isna() to get True/False mask of missing cells.
- Use .sum() on mask to count missing per column.
- True counts as 1, False as 0 in sum.
- Use .sum().sum() for total missing in DataFrame.
- Useful to find data quality issues quickly.

Full Transcript

We start with a DataFrame containing some missing values. Using df.isna(), we create a mask showing True where values are missing and False otherwise. Then, summing this mask per column counts how many missing values each column has. This works because True counts as 1 and False as 0 in sums. The final result is a count of missing values per column. This method helps quickly identify where data is incomplete.