Identifying missing values (isnull, isna) in Data Analysis Python - Cheat Sheet & Quick Revision

Recall & Review

beginner

What do the functions isnull() and isna() do in pandas?

Both isnull() and isna() check for missing values in a DataFrame or Series. They return a new object of the same shape with True where the data is missing and False where it is present.

beginner

How can you use isnull() to find how many missing values are in each column of a DataFrame?

You can call df.isnull().sum(). This returns a Series with the count of missing values for each column in the DataFrame df.

beginner

Are isnull() and isna() different in pandas?

No, they are exactly the same. isna() is just an alias for isnull(). You can use either one to detect missing values.

beginner

What type of values do isnull() and isna() detect as missing?

They detect NaN (Not a Number), None, and other null-like values as missing in pandas objects.

intermediate

How would you filter rows in a DataFrame to keep only those with missing values in a specific column?

Use df[df['column_name'].isnull()]. This returns rows where the specified column has missing values.

Which pandas function can you use to detect missing values in a DataFrame?

A dropna()

B fillna()

C replace()

D isnull()

isnull() returns a boolean mask showing missing values. fillna() fills missing values, dropna() removes them, and replace() replaces values.

What does df.isnull().sum() return?

A Count of missing values per column

B Total number of missing values in the DataFrame

C Rows with missing values

D DataFrame without missing values

It returns a Series with the count of missing values for each column.

Are isnull() and isna() different functions?

A No, they are aliases and behave the same

B No, but one is deprecated

C Yes, one works only on Series

D Yes, they detect different types of missing values

isna() is an alias for isnull(). Both behave identically.

Which of these values is detected as missing by isnull()?

A "" (empty string)

B NaN

C 0

D False

NaN is recognized as missing. Zero, empty string, and False are not missing.

How to select rows where column 'Age' has missing values?

A df[df['Age'].notnull()]

B df['Age'].fillna()

C df[df['Age'].isnull()]

D df.dropna(subset=['Age'])

Using df[df['Age'].isnull()] filters rows where 'Age' is missing.

Explain how to identify missing values in a pandas DataFrame using isnull() or isna().

Think about how to get a True/False map and then count Trues.

Describe how to filter rows in a DataFrame to keep only those with missing data in a specific column.

Focus on selecting rows where a column is missing.

Practice

1. What do the isnull() and isna() functions do in pandas?

easy

A. They identify missing or empty values in a DataFrame or Series.

B. They fill missing values with zeros.

C. They remove rows with missing values.

D. They convert data types to numeric.

Solution

1. Step 1: Understand the purpose of isnull() and isna()

   Both functions check for missing data and return True where data is missing.

2. Step 2: Differentiate from other functions

   They do not fill, remove, or convert data but only detect missing values.

3. Final Answer:

   They identify missing or empty values in a DataFrame or Series. -> Option A

4. Quick Check:

   Missing data detection = C [OK]

Hint: Think: True means missing, False means present [OK]

Common Mistakes:

• Confusing detection with filling missing values
• Thinking they remove data instead of marking it
• Assuming they change data types

2. Which of the following is the correct way to check for missing values in a pandas DataFrame named df?

easy

A. df.isempty()

B. df.checknull()

C. df.isnull()

D. df.missing()

Solution

1. Step 1: Recall pandas function names for missing data

   The correct pandas function to detect missing values is isnull().

2. Step 2: Verify other options

   Functions like checknull(), missing(), and isempty() do not exist in pandas for this purpose.

3. Final Answer:

   df.isnull() -> Option C

4. Quick Check:

   Correct pandas method = D [OK]

Hint: Remember: pandas uses isnull() or isna() [OK]

Common Mistakes:

• Using non-existent function names
• Confusing with methods that fill or drop data
• Misspelling function names

3. Given the following code, what is the output?

import pandas as pd
data = {'A': [1, None, 3], 'B': [4, 5, None]}
df = pd.DataFrame(data)
print(df.isna())

medium

A. A DataFrame with all False values.

B. A DataFrame showing True where values are missing and False otherwise.

C. A DataFrame with all True values.

D. An error because isna() is not a valid function.

Solution

1. Step 1: Understand the input DataFrame

   The DataFrame has missing values at df['A'][1] and df['B'][2].

2. Step 2: Apply isna() method

   This method returns True at missing positions and False elsewhere.

3. Final Answer:

   A DataFrame showing True where values are missing and False otherwise. -> Option B

4. Quick Check:

   Missing spots = True, others = False [OK]

Hint: Look for None or NaN spots turning True [OK]

Common Mistakes:

• Thinking isna() fills missing values
• Expecting all True or all False output
• Confusing isna() with dropna()

4. What is wrong with this code snippet to detect missing values?

import pandas as pd
data = {'X': [10, 20, None]}
df = pd.DataFrame(data)
print(df.isnull)

medium

A. The variable name df is invalid.

B. The DataFrame is not created correctly.

C. The isnull function does not exist in pandas.

D. The code is missing parentheses after isnull to call the function.

Solution

1. Step 1: Check the usage of isnull

   isnull is a function and needs parentheses to execute, like df.isnull().

2. Step 2: Verify other parts of the code

   DataFrame creation and variable names are correct; isnull exists in pandas.

3. Final Answer:

   The code is missing parentheses after isnull to call the function. -> Option D

4. Quick Check:

   Function call needs () = A [OK]

Hint: Remember to add () to call functions [OK]

Common Mistakes:

• Forgetting parentheses on function calls
• Assuming isnull is an attribute, not a method
• Misnaming variables or functions

5. You have a DataFrame df with columns ['Name', 'Age', 'Score']. Some Age and Score values are missing. How do you create a new column 'MissingCount' that counts how many missing values each row has?

hard

A. df['MissingCount'] = df.isnull().sum(axis=1)

B. df['MissingCount'] = df.isnull().sum()

C. df['MissingCount'] = df.isnull().count(axis=1)

D. df['MissingCount'] = df.isnull().mean(axis=1)

Solution

1. Step 1: Use isnull() to detect missing values

   This returns True/False for each cell indicating missing data.

2. Step 2: Sum missing values per row with sum(axis=1)

   Summing True values (counted as 1) across columns gives missing count per row.

3. Final Answer:

   df['MissingCount'] = df.isnull().sum(axis=1) -> Option A

4. Quick Check:

   Sum True per row = B [OK]

Hint: Sum isnull() across columns with axis=1 [OK]

Common Mistakes:

• Summing without axis=1 sums all missing in DataFrame
• Using count() instead of sum() to count missing
• Using mean() which gives fraction, not count