Identifying missing values (isnull, isna) in Data Analysis Python - Step-by-Step Execution

Concept Flow - Identifying missing values (isnull, isna)

Start with DataFrame

↓

Call isnull() or isna()

↓

Check each cell for missing value

↓

Return Boolean DataFrame

↓

Use result for analysis or cleaning

We start with a data table, check each cell for missing values using isnull() or isna(), then get a table of True/False showing missing spots.

Execution Sample

Data Analysis Python

import pandas as pd

df = pd.DataFrame({
  'A': [1, None, 3],
  'B': [4, 5, None]
})

missing = df.isnull()

This code creates a table with some missing values and then finds where those missing values are.

Execution Table

Step	DataFrame Cell	Value	Check isnull()	Result (True=missing)
1	df['A'][0]	1	isnull(1)	False
2	df['A'][1]	None	isnull(None)	True
3	df['A'][2]	3	isnull(3)	False
4	df['B'][0]	4	isnull(4)	False
5	df['B'][1]	5	isnull(5)	False
6	df['B'][2]	None	isnull(None)	True
7	End	-	-	All cells checked

💡 All cells checked for missing values, process ends.

Variable Tracker

Variable	Start	After Step 2	After Step 6	Final
df	{'A':[1,None,3],'B':[4,5,None]}	Same	Same	Same
missing	Not created	Partial (after step 2)	Partial (after step 6)	{'A':[False,True,False],'B':[False,False,True]}

Key Moments - 2 Insights

Why does isnull() return True for None but False for numbers?

Because None is the standard missing value marker in pandas, so isnull() flags it as True. Numbers are valid data, so they return False (see execution_table rows 1,2).

Are isnull() and isna() different in output?

No, both functions do the same check for missing values and return the same Boolean DataFrame (they are interchangeable).

Visual Quiz - 3 Questions

Look at the execution table, what is the result of isnull() for df['B'][2]?

A True

B False

C None

D Error

💡 Hint

Check execution_table row 6 for df['B'][2] value and isnull() result.

At which step does the first missing value get detected?

A Step 1

B Step 2

C Step 4

D Step 6

💡 Hint

Look at execution_table rows 1 and 2 to see when True first appears.

If the value None in df['A'][1] was replaced by 0, what would be the isnull() result at step 2?

A True

B None

C False

D Error

💡 Hint

Recall that isnull() returns False for valid numbers like 0 (see execution_table row 1).

Concept Snapshot

Use df.isnull() or df.isna() to find missing values in a DataFrame.
Returns a Boolean DataFrame with True where data is missing.
Missing values like None or NaN are flagged True.
Useful for cleaning or analyzing incomplete data.

Full Transcript

We start with a DataFrame containing some missing values marked as None. Calling isnull() checks each cell and returns True if the cell is missing data, otherwise False. The output is a Boolean DataFrame showing exactly where data is missing. This helps us find and handle missing data easily. Both isnull() and isna() do the same check and can be used interchangeably.

Practice

1. What do the isnull() and isna() functions do in pandas?

easy

A. They identify missing or empty values in a DataFrame or Series.

B. They fill missing values with zeros.

C. They remove rows with missing values.

D. They convert data types to numeric.

Solution

1. Step 1: Understand the purpose of isnull() and isna()

   Both functions check for missing data and return True where data is missing.

2. Step 2: Differentiate from other functions

   They do not fill, remove, or convert data but only detect missing values.

3. Final Answer:

   They identify missing or empty values in a DataFrame or Series. -> Option A

4. Quick Check:

   Missing data detection = C [OK]

Hint: Think: True means missing, False means present [OK]

Common Mistakes:

• Confusing detection with filling missing values
• Thinking they remove data instead of marking it
• Assuming they change data types

2. Which of the following is the correct way to check for missing values in a pandas DataFrame named df?

easy

A. df.isempty()

B. df.checknull()

C. df.isnull()

D. df.missing()

Solution

1. Step 1: Recall pandas function names for missing data

   The correct pandas function to detect missing values is isnull().

2. Step 2: Verify other options

   Functions like checknull(), missing(), and isempty() do not exist in pandas for this purpose.

3. Final Answer:

   df.isnull() -> Option C

4. Quick Check:

   Correct pandas method = D [OK]

Hint: Remember: pandas uses isnull() or isna() [OK]

Common Mistakes:

• Using non-existent function names
• Confusing with methods that fill or drop data
• Misspelling function names

3. Given the following code, what is the output?

import pandas as pd
data = {'A': [1, None, 3], 'B': [4, 5, None]}
df = pd.DataFrame(data)
print(df.isna())

medium

A. A DataFrame with all False values.

B. A DataFrame showing True where values are missing and False otherwise.

C. A DataFrame with all True values.

D. An error because isna() is not a valid function.

Solution

1. Step 1: Understand the input DataFrame

   The DataFrame has missing values at df['A'][1] and df['B'][2].

2. Step 2: Apply isna() method

   This method returns True at missing positions and False elsewhere.

3. Final Answer:

   A DataFrame showing True where values are missing and False otherwise. -> Option B

4. Quick Check:

   Missing spots = True, others = False [OK]

Hint: Look for None or NaN spots turning True [OK]

Common Mistakes:

• Thinking isna() fills missing values
• Expecting all True or all False output
• Confusing isna() with dropna()

4. What is wrong with this code snippet to detect missing values?

import pandas as pd
data = {'X': [10, 20, None]}
df = pd.DataFrame(data)
print(df.isnull)

medium

A. The variable name df is invalid.

B. The DataFrame is not created correctly.

C. The isnull function does not exist in pandas.

D. The code is missing parentheses after isnull to call the function.

Solution

1. Step 1: Check the usage of isnull

   isnull is a function and needs parentheses to execute, like df.isnull().

2. Step 2: Verify other parts of the code

   DataFrame creation and variable names are correct; isnull exists in pandas.

3. Final Answer:

   The code is missing parentheses after isnull to call the function. -> Option D

4. Quick Check:

   Function call needs () = A [OK]

Hint: Remember to add () to call functions [OK]

Common Mistakes:

• Forgetting parentheses on function calls
• Assuming isnull is an attribute, not a method
• Misnaming variables or functions

5. You have a DataFrame df with columns ['Name', 'Age', 'Score']. Some Age and Score values are missing. How do you create a new column 'MissingCount' that counts how many missing values each row has?

hard

A. df['MissingCount'] = df.isnull().sum(axis=1)

B. df['MissingCount'] = df.isnull().sum()

C. df['MissingCount'] = df.isnull().count(axis=1)

D. df['MissingCount'] = df.isnull().mean(axis=1)

Solution

1. Step 1: Use isnull() to detect missing values

   This returns True/False for each cell indicating missing data.

2. Step 2: Sum missing values per row with sum(axis=1)

   Summing True values (counted as 1) across columns gives missing count per row.

3. Final Answer:

   df['MissingCount'] = df.isnull().sum(axis=1) -> Option A

4. Quick Check:

   Sum True per row = B [OK]

Hint: Sum isnull() across columns with axis=1 [OK]

Common Mistakes:

• Summing without axis=1 sums all missing in DataFrame
• Using count() instead of sum() to count missing
• Using mean() which gives fraction, not count