Counting missing values in Pandas - Mini Project: Build & Apply

Counting missing values

📖 Scenario: You work as a data analyst. You have a small table of sales data. Some values are missing because the data was not recorded properly.You want to find out how many missing values are in the data to understand its quality.

🎯 Goal: Create a pandas DataFrame with sales data including some missing values. Then count how many missing values are in each column.

📋 What You'll Learn

Use pandas to create a DataFrame

Include missing values using None or numpy.nan

Count missing values per column using pandas methods

Print the count of missing values

💡 Why This Matters

🌍 Real World

Data scientists often check for missing data to decide how to clean or fill it before analysis.

💼 Career

Counting missing values is a basic but essential skill for data cleaning in data analyst and data scientist roles.

1

Create the sales data DataFrame

Import pandas as pd and create a DataFrame called sales_data with these exact columns and values:
'Product': ['Apple', 'Banana', 'Orange', 'Grapes'],
'Price': [1.2, None, 0.8, 2.5],
'Quantity': [10, 15, None, 8].

Pandas

# Import pandas and create the DataFrame sales_data with the given values
# Your code here

Hint

Use pd.DataFrame with a dictionary where keys are column names and values are lists of column data.

2

Create a variable to hold missing values count

Create a variable called missing_counts and set it to None. This will hold the count of missing values later.

Pandas

import pandas as pd

sales_data = pd.DataFrame({
    'Product': ['Apple', 'Banana', 'Orange', 'Grapes'],
    'Price': [1.2, None, 0.8, 2.5],
    'Quantity': [10, 15, None, 8]
})

# Create a variable missing_counts and set it to None
# Your code here

Hint

Just write missing_counts = None to prepare the variable.

3

Count missing values in each column

Set the variable missing_counts to the result of calling sales_data.isnull().sum(). This counts missing values per column.

Pandas

import pandas as pd

sales_data = pd.DataFrame({
    'Product': ['Apple', 'Banana', 'Orange', 'Grapes'],
    'Price': [1.2, None, 0.8, 2.5],
    'Quantity': [10, 15, None, 8]
})

missing_counts = None

# Count missing values per column and assign to missing_counts
# Your code here

Hint

Use sales_data.isnull() to find missing values, then .sum() to count them per column.

4

Print the missing values count

Print the variable missing_counts to show the count of missing values in each column.

Pandas

import pandas as pd

sales_data = pd.DataFrame({
    'Product': ['Apple', 'Banana', 'Orange', 'Grapes'],
    'Price': [1.2, None, 0.8, 2.5],
    'Quantity': [10, 15, None, 8]
})

missing_counts = sales_data.isnull().sum()

# Print the missing_counts variable
# Your code here

Hint

Use print(missing_counts) to display the counts.

Practice

1. What does the pandas method df.isna().sum() do when applied to a DataFrame df?

easy

A. Counts the total number of rows in the DataFrame

B. Counts the number of missing values in each column of the DataFrame

C. Replaces missing values with zero in the DataFrame

D. Returns the data types of each column in the DataFrame

Solution

1. Step 1: Understand df.isna()

   This method returns a DataFrame of the same shape as df with True where values are missing and False otherwise.

2. Step 2: Apply .sum() on the boolean DataFrame

   Summing True values (which count as 1) per column gives the count of missing values in each column.

3. Final Answer:

   Counts the number of missing values in each column of the DataFrame -> Option B

4. Quick Check:

   df.isna().sum() counts missing per column [OK]

Hint: Use isna() then sum() to count missing per column [OK]

Common Mistakes:

• Thinking sum() counts rows instead of missing values
• Confusing isna() with fillna()
• Assuming it replaces missing values

2. Which of the following is the correct syntax to count all missing values in a pandas DataFrame df?

easy

A. df.isna().count()

B. df.count().sum()

C. df.isna().sum().sum()

D. df.isnull().count()

Solution

1. Step 1: Use df.isna() or df.isnull() to find missing values

   Both return a boolean DataFrame marking missing values as True.

2. Step 2: Use .sum().sum() to count all missing values

   The first sum() sums missing per column, the second sums those counts to get total missing in the DataFrame.

3. Final Answer:

   df.isna().sum().sum() -> Option C

4. Quick Check:

   Total missing = df.isna().sum().sum() [OK]

Hint: Double sum() after isna() counts all missing values [OK]

Common Mistakes:

• Using count() which counts non-missing values
• Using isna().count() which counts all rows, not missing
• Confusing isna() with isnull().count()

3. Given the DataFrame df below:

import pandas as pd
data = {'A': [1, None, 3], 'B': [None, None, 6], 'C': [7, 8, 9]}
df = pd.DataFrame(data)

What is the output of df.isna().sum()?

medium

A. A 1
B 1
C 1
dtype: int64

B. A 0
B 0
C 0
dtype: int64

C. A 2
B 1
C 0
dtype: int64

D. A 1
B 2
C 0
dtype: int64

Solution

1. Step 1: Identify missing values in each column

   Column A has one None (missing), column B has two None values, column C has none.

2. Step 2: Apply df.isna().sum()

   This counts missing per column: A=1, B=2, C=0.

3. Final Answer:

   A 1 B 2 C 0 dtype: int64 -> Option D

4. Quick Check:

   Count missing per column matches output [OK]

Hint: Count None or NaN per column to verify sums [OK]

Common Mistakes:

• Mixing counts of missing and non-missing
• Miscounting missing in column B
• Assuming None counts as zero

4. You run the code below but get an error:

import pandas as pd
data = {'X': [1, 2, None], 'Y': [None, 5, 6]}
df = pd.DataFrame(data)
missing = df.isna.sum()

What is the error and how to fix it?

medium

A. Missing parentheses after isna; fix with df.isna().sum()

B. DataFrame missing columns; add columns before counting

C. sum() is not a pandas method; use count() instead

D. None values cause error; replace with zeros first

Solution

1. Step 1: Identify the error in method call

   The code uses df.isna.sum() missing parentheses after isna, which is a method.

2. Step 2: Correct the syntax

   Use df.isna().sum() to call the method and then sum missing values.

3. Final Answer:

   Missing parentheses after isna; fix with df.isna().sum() -> Option A

4. Quick Check:

   Method calls need parentheses [OK]

Hint: Remember to add () after isna to call the method [OK]

Common Mistakes:

• Forgetting parentheses on isna()
• Using sum() instead of count() incorrectly
• Thinking None causes errors in counting

5. You have a DataFrame df with columns ['age', 'income', 'score']. You want to find the percentage of missing values in each column. Which code snippet correctly calculates this?

hard

A. df.isna().sum() / len(df) * 100

B. df.isna().count() / len(df) * 100

C. df.count() / df.isna().sum() * 100

D. df.isna().sum() * 100

Solution

1. Step 1: Count missing values per column

   Use df.isna().sum() to get missing counts per column.

2. Step 2: Calculate percentage of missing values

   Divide missing counts by total rows len(df) and multiply by 100 to get percentage.

3. Final Answer:

   df.isna().sum() / len(df) * 100 -> Option A

4. Quick Check:

   Missing % = (missing count / total rows) * 100 [OK]

Hint: Divide missing count by total rows, multiply by 100 [OK]

Common Mistakes:

• Using count() which counts non-missing values
• Multiplying missing count by 100 without dividing
• Dividing count by missing count incorrectly