Detecting missing values with isna() in Pandas - Step-by-Step Execution

Concept Flow - Detecting missing values with isna()

Start with DataFrame

↓

Call isna() method

↓

Check each cell for missing value

↓

Return DataFrame of True/False

↓

Use result to find or handle missing data

We start with a DataFrame, call isna() to check each cell for missing values, and get back a DataFrame showing True where data is missing and False otherwise.

Execution Sample

Pandas

import pandas as pd

data = {'A': [1, None, 3], 'B': [4, 5, None]}
df = pd.DataFrame(data)
missing = df.isna()
print(missing)

This code creates a DataFrame with some missing values and uses isna() to detect them, printing a DataFrame of True/False.

Execution Table

Step	DataFrame Cell	Value	isna() Result	Explanation
1	df['A'][0]	1	False	Value is 1, not missing
2	df['A'][1]	None	True	Value is None, missing detected
3	df['A'][2]	3	False	Value is 3, not missing
4	df['B'][0]	4	False	Value is 4, not missing
5	df['B'][1]	5	False	Value is 5, not missing
6	df['B'][2]	None	True	Value is None, missing detected
7	End	-	-	All cells checked, isna() returns DataFrame of True/False

💡 All cells checked, isna() returns DataFrame showing True where values are missing.

Variable Tracker

Variable	Start	After isna() call
df	{'A':[1,None,3],'B':[4,5,None]}	Same DataFrame, unchanged
missing	Not defined	{'A':[False,True,False],'B':[False,False,True]}

Key Moments - 2 Insights

Why does isna() return True for None values?

Because None represents missing data in pandas, isna() detects it and returns True as shown in rows 2 and 6 of the execution_table.

Does isna() change the original DataFrame?

No, isna() returns a new DataFrame of booleans without modifying the original, as shown in variable_tracker where df stays the same.

Visual Quiz - 3 Questions

Look at the execution_table, what is the isna() result for df['B'][1]?

A True

B False

C None

D Error

💡 Hint

Check row 5 in the execution_table where df['B'][1] is 5 and isna() returns False.

At which step does isna() detect the first missing value?

A Step 1

B Step 4

C Step 2

D Step 6

💡 Hint

Look at execution_table row 2 where df['A'][1] is None and isna() returns True.

If the DataFrame had no None values, what would the missing DataFrame contain?

A All False

B All True

C Mixed True and False

D Error

💡 Hint

Refer to variable_tracker and execution_table showing True only where values are None.

Concept Snapshot

pandas.DataFrame.isna() detects missing values.
Returns a DataFrame of booleans: True if missing, False if not.
Does not change original data.
Use to find or filter missing data easily.

Full Transcript

We start with a pandas DataFrame containing some missing values represented by None. Calling the isna() method checks each cell and returns a new DataFrame of the same shape with True where values are missing and False otherwise. This helps us quickly see where data is missing without changing the original DataFrame. For example, in the sample code, isna() returns True for cells with None and False for others. This method is useful for detecting missing data before cleaning or analysis.

Practice

1. What does the pandas.DataFrame.isna() function do?

easy

A. It converts all values to strings.

B. It fills missing values with zeros.

C. It removes rows with missing values.

D. It shows which values in the DataFrame are missing (NaN).

Solution

1. Step 1: Understand the purpose of isna()

   The isna() function checks each cell in a DataFrame and marks True if the value is missing (NaN), otherwise False.

2. Step 2: Compare options with function behavior

   Only It shows which values in the DataFrame are missing (NaN). correctly describes this behavior. Options B, C, and D describe other functions or actions.

3. Final Answer:

   It shows which values in the DataFrame are missing (NaN). -> Option D

4. Quick Check:

   isna() = marks missing values True [OK]

Hint: Remember: isna() marks missing spots True [OK]

Common Mistakes:

• Confusing isna() with fillna()
• Thinking isna() removes data
• Assuming isna() changes data types

2. Which of the following is the correct syntax to check missing values in a DataFrame named df?

easy

A. isna(df)

B. df.isna

C. df.isna()

D. df.isna[]

Solution

1. Step 1: Recall method call syntax in pandas

   Methods like isna() must be called with parentheses on the DataFrame object, so df.isna() is correct.

2. Step 2: Evaluate each option

   df.isna() uses correct syntax. df.isna misses parentheses, so it returns the method object, not the result. isna(df) is incorrect because isna is a method, not a standalone function. df.isna[] uses invalid bracket syntax.

3. Final Answer:

   df.isna() -> Option C

4. Quick Check:

   Method call needs parentheses () [OK]

Hint: Use parentheses to call isna(): df.isna() [OK]

Common Mistakes:

• Forgetting parentheses after isna
• Trying to call isna as a standalone function
• Using brackets instead of parentheses

3. Given the DataFrame:

import pandas as pd
data = {'A': [1, None, 3], 'B': [4, 5, None]}
df = pd.DataFrame(data)
print(df.isna())

What is the output?

medium

A. A B
0 False False
1 True False
2 False True

B. A B
0 True False
1 False True
2 False False

C. A B
0 False True
1 False False
2 True False

D. A B
0 False False
1 False True
2 True False

Solution

1. Step 1: Identify missing values in each column

   Column 'A' has None at index 1, so True there. Column 'B' has None at index 2, so True there. All other values are present (False).

2. Step 2: Match the True/False pattern to output

   A B 0 False False 1 True False 2 False True matches the pattern: index 1 in 'A' is True, index 2 in 'B' is True, others False.

3. Final Answer:

   A B 0 False False 1 True False 2 False True -> Option A

4. Quick Check:

   None values marked True by isna() [OK]

Hint: Look for None positions; isna() marks them True [OK]

Common Mistakes:

• Mixing row and column positions
• Confusing True and False values
• Assuming isna() replaces values

4. You run this code:

import pandas as pd
data = {'X': [10, 20, None], 'Y': [None, 40, 50]}
df = pd.DataFrame(data)
missing = df.isna

Why does missing.sum() cause an error?

medium

A. Because missing is a DataFrame, not a function.

B. Because isna is not called with parentheses when assigned.

C. Because isna() does not exist in pandas.

D. Because df is not defined.

Solution

1. Step 1: Understand method assignment without parentheses

   Assigning missing = df.isna stores the method itself, not the result. To get the result, you must call it with parentheses.

2. Step 2: Explain why missing.sum() causes error

   Assigning missing = df.isna stores the method object, not the DataFrame result. Method objects do not have a .sum() attribute--only DataFrames do. Thus, missing.sum() raises AttributeError.

3. Final Answer:

   Because isna is not called with parentheses when assigned. -> Option B

4. Quick Check:

   Assign method with () to get result [OK]

Hint: Assign with parentheses to get result, not method [OK]

Common Mistakes:

• Assigning method without parentheses and expecting result
• Confusing method object with method call
• Assuming isna() is not a pandas method

5. You have a DataFrame df with columns 'Age' and 'Salary'. You want to count how many missing values are in each column using isna(). Which code correctly does this?

hard

A. df.isna().sum()

B. df.isna.sum()

C. df.sum().isna()

D. df.isna().count()

Solution

1. Step 1: Use isna() to mark missing values

   The isna() method returns True for missing values and False otherwise.

2. Step 2: Sum True values per column to count missing

   Since True is treated as 1 and False as 0, summing over rows counts missing values per column. So df.isna().sum() gives the count per column.

3. Final Answer:

   df.isna().sum() -> Option A

4. Quick Check:

   Sum True from isna() to count missing [OK]

Hint: Sum isna() results to count missing per column [OK]

Common Mistakes:

• Forgetting parentheses after isna
• Using count() which counts all rows, not missing
• Applying sum() before isna()