Detecting missing values with isna() in Pandas - Time & Space Complexity

Time Complexity: Detecting missing values with isna()

O(n * m)

Understanding Time Complexity

We want to understand how the time to find missing values changes as data grows.

How does checking for missing data scale with more rows?

Scenario Under Consideration

Analyze the time complexity of the following code snippet.

import pandas as pd

data = pd.DataFrame({
    'A': [1, 2, None, 4],
    'B': [None, 2, 3, 4],
    'C': [1, None, None, 4]
})

missing_mask = data.isna()

This code creates a table and checks each cell to see if it is missing (NaN).

Identify Repeating Operations

Identify the loops, recursion, array traversals that repeat.

• Primary operation: Checking each cell in the DataFrame for missing value.
• How many times: Once per cell, so number of rows times number of columns.

How Execution Grows With Input

As the table gets bigger, the number of checks grows with total cells.

Input Size (rows x columns)	Approx. Operations
10 x 5 = 50	50 checks
100 x 5 = 500	500 checks
1000 x 5 = 5000	5000 checks

Pattern observation: The work grows directly with the number of cells.

Final Time Complexity

Time Complexity: O(n * m)

This means the time grows proportionally with the number of rows (n) times columns (m).

Common Mistake

[X] Wrong: "Checking for missing values is constant time regardless of data size."

[OK] Correct: Each cell must be checked, so more data means more work.

Interview Connect

Understanding how data size affects missing value detection helps you reason about data cleaning speed in real projects.

Self-Check

"What if we only check one column for missing values instead of the whole DataFrame? How would the time complexity change?"

Practice

1. What does the pandas.DataFrame.isna() function do?

easy

A. It converts all values to strings.

B. It fills missing values with zeros.

C. It removes rows with missing values.

D. It shows which values in the DataFrame are missing (NaN).

Solution

1. Step 1: Understand the purpose of isna()

   The isna() function checks each cell in a DataFrame and marks True if the value is missing (NaN), otherwise False.

2. Step 2: Compare options with function behavior

   Only It shows which values in the DataFrame are missing (NaN). correctly describes this behavior. Options B, C, and D describe other functions or actions.

3. Final Answer:

   It shows which values in the DataFrame are missing (NaN). -> Option D

4. Quick Check:

   isna() = marks missing values True [OK]

Hint: Remember: isna() marks missing spots True [OK]

Common Mistakes:

• Confusing isna() with fillna()
• Thinking isna() removes data
• Assuming isna() changes data types

2. Which of the following is the correct syntax to check missing values in a DataFrame named df?

easy

A. isna(df)

B. df.isna

C. df.isna()

D. df.isna[]

Solution

1. Step 1: Recall method call syntax in pandas

   Methods like isna() must be called with parentheses on the DataFrame object, so df.isna() is correct.

2. Step 2: Evaluate each option

   df.isna() uses correct syntax. df.isna misses parentheses, so it returns the method object, not the result. isna(df) is incorrect because isna is a method, not a standalone function. df.isna[] uses invalid bracket syntax.

3. Final Answer:

   df.isna() -> Option C

4. Quick Check:

   Method call needs parentheses () [OK]

Hint: Use parentheses to call isna(): df.isna() [OK]

Common Mistakes:

• Forgetting parentheses after isna
• Trying to call isna as a standalone function
• Using brackets instead of parentheses

3. Given the DataFrame:

import pandas as pd
data = {'A': [1, None, 3], 'B': [4, 5, None]}
df = pd.DataFrame(data)
print(df.isna())

What is the output?

medium

A. A B
0 False False
1 True False
2 False True

B. A B
0 True False
1 False True
2 False False

C. A B
0 False True
1 False False
2 True False

D. A B
0 False False
1 False True
2 True False

Solution

1. Step 1: Identify missing values in each column

   Column 'A' has None at index 1, so True there. Column 'B' has None at index 2, so True there. All other values are present (False).

2. Step 2: Match the True/False pattern to output

   A B 0 False False 1 True False 2 False True matches the pattern: index 1 in 'A' is True, index 2 in 'B' is True, others False.

3. Final Answer:

   A B 0 False False 1 True False 2 False True -> Option A

4. Quick Check:

   None values marked True by isna() [OK]

Hint: Look for None positions; isna() marks them True [OK]

Common Mistakes:

• Mixing row and column positions
• Confusing True and False values
• Assuming isna() replaces values

4. You run this code:

import pandas as pd
data = {'X': [10, 20, None], 'Y': [None, 40, 50]}
df = pd.DataFrame(data)
missing = df.isna

Why does missing.sum() cause an error?

medium

A. Because missing is a DataFrame, not a function.

B. Because isna is not called with parentheses when assigned.

C. Because isna() does not exist in pandas.

D. Because df is not defined.

Solution

1. Step 1: Understand method assignment without parentheses

   Assigning missing = df.isna stores the method itself, not the result. To get the result, you must call it with parentheses.

2. Step 2: Explain why missing.sum() causes error

   Assigning missing = df.isna stores the method object, not the DataFrame result. Method objects do not have a .sum() attribute--only DataFrames do. Thus, missing.sum() raises AttributeError.

3. Final Answer:

   Because isna is not called with parentheses when assigned. -> Option B

4. Quick Check:

   Assign method with () to get result [OK]

Hint: Assign with parentheses to get result, not method [OK]

Common Mistakes:

• Assigning method without parentheses and expecting result
• Confusing method object with method call
• Assuming isna() is not a pandas method

5. You have a DataFrame df with columns 'Age' and 'Salary'. You want to count how many missing values are in each column using isna(). Which code correctly does this?

hard

A. df.isna().sum()

B. df.isna.sum()

C. df.sum().isna()

D. df.isna().count()

Solution

1. Step 1: Use isna() to mark missing values

   The isna() method returns True for missing values and False otherwise.

2. Step 2: Sum True values per column to count missing

   Since True is treated as 1 and False as 0, summing over rows counts missing values per column. So df.isna().sum() gives the count per column.

3. Final Answer:

   df.isna().sum() -> Option A

4. Quick Check:

   Sum True from isna() to count missing [OK]

Hint: Sum isna() results to count missing per column [OK]

Common Mistakes:

• Forgetting parentheses after isna
• Using count() which counts all rows, not missing
• Applying sum() before isna()