How to Check for Missing Values in pandas DataFrames

To check for missing values in pandas, use the isna() or isnull() methods which return a DataFrame of True/False indicating missing data. You can then use sum() to count missing values per column or row.

Syntax

The main methods to check missing values in pandas are:

• DataFrame.isna(): Returns a DataFrame of booleans where True means the value is missing.
• DataFrame.isnull(): Alias for isna(), works the same way.
• DataFrame.sum(): When used after isna(), it counts the number of missing values per column or row.

python

df.isna()
df.isnull()
df.isna().sum()

Example

This example shows how to create a DataFrame with missing values and check them using isna() and sum().

python

import pandas as pd

data = {'Name': ['Alice', 'Bob', None, 'David'],
        'Age': [25, None, 30, 22],
        'City': ['New York', 'Los Angeles', 'Chicago', None]}

df = pd.DataFrame(data)

# Check missing values (True means missing)
missing_values = df.isna()

# Count missing values per column
missing_count = df.isna().sum()

print("Missing values DataFrame:")
print(missing_values)
print("\nCount of missing values per column:")
print(missing_count)

Output

Missing values DataFrame:
    Name    Age   City
0  False  False  False
1  False   True  False
2   True  False  False
3  False  False   True

Count of missing values per column:
Name    1
Age     1
City    1
dtype: int64

Common Pitfalls

Some common mistakes when checking for missing values:

• Using == None or == np.nan to check missing values does not work because np.nan is not equal to itself.
• Confusing isna() with notna(). The latter returns True for non-missing values.
• For counting missing values, forgetting to use sum() after isna() returns a boolean DataFrame instead of counts.

Correct way to check missing values:

python

import numpy as np
import pandas as pd

df = pd.DataFrame({'A': [1, np.nan, 3]})

# Wrong way (returns False always)
print(df['A'] == np.nan)

# Right way
print(df['A'].isna())

Output

[False False False]
0    False
1     True
2    False
Name: A, dtype: bool

Quick Reference

Summary of methods to check missing values in pandas:

Method	Description	Returns
isna()	Detects missing values	Boolean DataFrame/Series
isnull()	Alias for isna()	Boolean DataFrame/Series
notna()	Detects non-missing values	Boolean DataFrame/Series
sum() after isna()	Counts missing values per column or row	Integer counts

Key Takeaways

Use isna() or isnull() to find missing values in pandas DataFrames.

Apply sum() on the boolean result to count missing values per column or row.

Never check missing values with == None or == np.nan as they don't work correctly.

Remember notna() returns True for non-missing values, useful for filtering.

Always verify your DataFrame for missing data before analysis to avoid errors.