Identifying missing values (isnull, isna) in Data Analysis Python - Mini Project: Build & Apply

Identifying Missing Values in Data

📖 Scenario: Imagine you work in a company that collects customer data. Sometimes, some information is missing. You want to find out which parts of the data are missing so you can fix or handle them.

🎯 Goal: You will create a small table of customer data, then use Python to find which values are missing using isnull() or isna() methods.

📋 What You'll Learn

Create a pandas DataFrame with specific customer data including some missing values

Create a variable to hold the result of checking for missing values

Use the isnull() method on the DataFrame to find missing values

Print the result showing which values are missing

💡 Why This Matters

🌍 Real World

In real life, data often has missing parts. Finding missing values helps you clean data before analysis.

💼 Career

Data scientists and analysts must identify and handle missing data to make accurate models and reports.

1

Create the customer data table

Import pandas as pd and create a DataFrame called customers with these exact data: three columns named 'Name', 'Age', and 'Email'. The rows should be: 'Alice', 25, '[e-mail]', 'Bob', None, '[e-mail]', and 'Charlie', 30, None.

Data Analysis Python

import pandas as pd
# Create the DataFrame called customers with the exact data
# Your code here

Hint

Use pd.DataFrame with a dictionary where keys are column names and values are lists of column data.

2

Create a variable to check for missing values

Create a variable called missing_values and set it to the result of calling isnull() on the customers DataFrame.

Data Analysis Python

import pandas as pd
customers = pd.DataFrame({
    'Name': ['Alice', 'Bob', 'Charlie'],
    'Age': [25, None, 30],
    'Email': ['[e-mail]', '[e-mail]', None]
})
# Create missing_values by calling isnull() on customers
# Your code here

Hint

Use customers.isnull() to get a DataFrame showing True where values are missing.

3

Use the core method to find missing values

Use the isnull() method on the customers DataFrame and assign the result to the variable missing_values.

Data Analysis Python

import pandas as pd
customers = pd.DataFrame({
    'Name': ['Alice', 'Bob', 'Charlie'],
    'Age': [25, None, 30],
    'Email': ['[e-mail]', '[e-mail]', None]
})
missing_values = customers.isnull()
# Your code here

Hint

This step repeats the core logic to reinforce the method usage.

4

Print the missing values table

Print the variable missing_values to show which values are missing in the customers DataFrame.

Data Analysis Python

import pandas as pd
customers = pd.DataFrame({
    'Name': ['Alice', 'Bob', 'Charlie'],
    'Age': [25, None, 30],
    'Email': ['[e-mail]', '[e-mail]', None]
})
missing_values = customers.isnull()
# Print missing_values below
# Your code here

Hint

Use print(missing_values) to display the table of True/False values.

Practice

1. What do the isnull() and isna() functions do in pandas?

easy

A. They identify missing or empty values in a DataFrame or Series.

B. They fill missing values with zeros.

C. They remove rows with missing values.

D. They convert data types to numeric.

Solution

1. Step 1: Understand the purpose of isnull() and isna()

   Both functions check for missing data and return True where data is missing.

2. Step 2: Differentiate from other functions

   They do not fill, remove, or convert data but only detect missing values.

3. Final Answer:

   They identify missing or empty values in a DataFrame or Series. -> Option A

4. Quick Check:

   Missing data detection = C [OK]

Hint: Think: True means missing, False means present [OK]

Common Mistakes:

• Confusing detection with filling missing values
• Thinking they remove data instead of marking it
• Assuming they change data types

2. Which of the following is the correct way to check for missing values in a pandas DataFrame named df?

easy

A. df.isempty()

B. df.checknull()

C. df.isnull()

D. df.missing()

Solution

1. Step 1: Recall pandas function names for missing data

   The correct pandas function to detect missing values is isnull().

2. Step 2: Verify other options

   Functions like checknull(), missing(), and isempty() do not exist in pandas for this purpose.

3. Final Answer:

   df.isnull() -> Option C

4. Quick Check:

   Correct pandas method = D [OK]

Hint: Remember: pandas uses isnull() or isna() [OK]

Common Mistakes:

• Using non-existent function names
• Confusing with methods that fill or drop data
• Misspelling function names

3. Given the following code, what is the output?

import pandas as pd
data = {'A': [1, None, 3], 'B': [4, 5, None]}
df = pd.DataFrame(data)
print(df.isna())

medium

A. A DataFrame with all False values.

B. A DataFrame showing True where values are missing and False otherwise.

C. A DataFrame with all True values.

D. An error because isna() is not a valid function.

Solution

1. Step 1: Understand the input DataFrame

   The DataFrame has missing values at df['A'][1] and df['B'][2].

2. Step 2: Apply isna() method

   This method returns True at missing positions and False elsewhere.

3. Final Answer:

   A DataFrame showing True where values are missing and False otherwise. -> Option B

4. Quick Check:

   Missing spots = True, others = False [OK]

Hint: Look for None or NaN spots turning True [OK]

Common Mistakes:

• Thinking isna() fills missing values
• Expecting all True or all False output
• Confusing isna() with dropna()

4. What is wrong with this code snippet to detect missing values?

import pandas as pd
data = {'X': [10, 20, None]}
df = pd.DataFrame(data)
print(df.isnull)

medium

A. The variable name df is invalid.

B. The DataFrame is not created correctly.

C. The isnull function does not exist in pandas.

D. The code is missing parentheses after isnull to call the function.

Solution

1. Step 1: Check the usage of isnull

   isnull is a function and needs parentheses to execute, like df.isnull().

2. Step 2: Verify other parts of the code

   DataFrame creation and variable names are correct; isnull exists in pandas.

3. Final Answer:

   The code is missing parentheses after isnull to call the function. -> Option D

4. Quick Check:

   Function call needs () = A [OK]

Hint: Remember to add () to call functions [OK]

Common Mistakes:

• Forgetting parentheses on function calls
• Assuming isnull is an attribute, not a method
• Misnaming variables or functions

5. You have a DataFrame df with columns ['Name', 'Age', 'Score']. Some Age and Score values are missing. How do you create a new column 'MissingCount' that counts how many missing values each row has?

hard

A. df['MissingCount'] = df.isnull().sum(axis=1)

B. df['MissingCount'] = df.isnull().sum()

C. df['MissingCount'] = df.isnull().count(axis=1)

D. df['MissingCount'] = df.isnull().mean(axis=1)

Solution

1. Step 1: Use isnull() to detect missing values

   This returns True/False for each cell indicating missing data.

2. Step 2: Sum missing values per row with sum(axis=1)

   Summing True values (counted as 1) across columns gives missing count per row.

3. Final Answer:

   df['MissingCount'] = df.isnull().sum(axis=1) -> Option A

4. Quick Check:

   Sum True per row = B [OK]

Hint: Sum isnull() across columns with axis=1 [OK]

Common Mistakes:

• Summing without axis=1 sums all missing in DataFrame
• Using count() instead of sum() to count missing
• Using mean() which gives fraction, not count