Data validation checks in Pandas - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Check for missing values in a DataFrame

What is the output of this code that checks for missing values in the DataFrame?

Pandas

import pandas as pd

df = pd.DataFrame({
    'A': [1, 2, None, 4],
    'B': ['x', None, 'y', 'z']
})

result = df.isnull().sum()

A

A    1
B    1
dtype: int64

B

A    0
B    0
dtype: int64

C

A    1
B    0
dtype: int64

D

A    0
B    1
dtype: int64

💡 Hint

Use the isnull() method to find missing values and sum() to count them per column.

The DataFrame has one missing value in column 'A' and one in column 'B'. The isnull() method marks missing values as True, and sum() counts them.

❓ data_output

intermediate

Count unique values per column

What is the output of this code that counts unique values in each column of the DataFrame?

Pandas

import pandas as pd

df = pd.DataFrame({
    'Color': ['red', 'blue', 'red', 'green'],
    'Shape': ['circle', 'square', 'circle', 'triangle']
})

unique_counts = df.nunique()

A

Color    4
Shape    4
dtype: int64

B

Color    3
Shape    3
dtype: int64

C

Color    3
Shape    4
dtype: int64

D

Color    4
Shape    3
dtype: int64

💡 Hint

The nunique() method counts distinct values per column.

The 'Color' column has three unique values: red, blue, green. The 'Shape' column has three unique values: circle, square, triangle.

🔧 Debug

advanced

Identify the error in data type validation

What error does this code raise when checking if all values in column 'Age' are integers?

Pandas

import pandas as pd

df = pd.DataFrame({'Age': [25, 30, 'thirty-five', 40]})

all_int = df['Age'].apply(lambda x: isinstance(x, int)).all()

A False

B True

C TypeError

D ValueError

💡 Hint

Check the data types of each value in the 'Age' column.

The third value is a string 'thirty-five', so the isinstance check returns False for that value, making all() return False.

🚀 Application

advanced

Detect duplicate rows in a DataFrame

Which option correctly returns a DataFrame containing only the duplicate rows?

Pandas

import pandas as pd

df = pd.DataFrame({
    'ID': [1, 2, 2, 3, 4, 4, 4],
    'Value': ['a', 'b', 'b', 'c', 'd', 'd', 'd']
})

A df[df.duplicated(keep='first')]

B df[df.duplicated()]

C df[df.duplicated(keep='last')]

D df[df.duplicated(keep=False)]

💡 Hint

Use duplicated() with keep=False to mark all duplicates as True.

duplicated(keep=False) marks all occurrences of duplicates as True, so filtering returns all duplicate rows.

🧠 Conceptual

expert

Understanding data validation with custom rules

You want to validate a DataFrame column 'Score' to ensure all values are between 0 and 100 inclusive. Which code snippet correctly returns True if all values meet this condition?

A (df['Score'] >= 0) & (df['Score'] <= 100).all()

B df['Score'].apply(lambda x: 0 <= x <= 100).all()

C df['Score'].between(0, 100).all()

D ((df['Score'] >= 0) & (df['Score'] <= 100)).all()

💡 Hint

Use the pandas between() method for inclusive range checks.

The between() method checks if values are between two bounds inclusive. Using .all() confirms all values meet the condition.

Practice

1. What does the pandas function isnull() check in a DataFrame?

easy

A. It checks for missing or null values in the data.

B. It checks for duplicate rows in the DataFrame.

C. It returns the data types of each column.

D. It sorts the DataFrame by a column.

Solution

1. Step 1: Understand the purpose of isnull()

   The isnull() function in pandas returns a DataFrame of the same shape with True where values are missing (NaN) and False otherwise.

2. Step 2: Compare with other functions

   Functions like duplicated() check duplicates, and dtypes shows data types, so they do not check for missing values.

3. Final Answer:

   It checks for missing or null values in the data. -> Option A

4. Quick Check:

   Missing data = isnull() [OK]

Hint: Missing values? Use isnull() to find them fast [OK]

Common Mistakes:

• Confusing isnull() with duplicated()
• Thinking isnull() returns data types
• Assuming isnull() sorts data

2. Which of the following is the correct syntax to check for duplicate rows in a pandas DataFrame named df?

easy

A. df.isnull()

B. df.check_duplicates()

C. df.duplicated()

D. df.find_duplicates()

Solution

1. Step 1: Recall pandas method for duplicates

   The correct pandas method to find duplicate rows is duplicated(), called as df.duplicated().

2. Step 2: Verify other options

   isnull() checks missing data, and check_duplicates() and find_duplicates() are not valid pandas methods.

3. Final Answer:

   df.duplicated() -> Option C

4. Quick Check:

   Duplicates = duplicated() [OK]

Hint: Duplicates? Use df.duplicated() to spot them [OK]

Common Mistakes:

• Using non-existent methods like check_duplicates()
• Mixing isnull() with duplicate checks
• Forgetting parentheses after duplicated

3. Given the DataFrame df below, what will df.isnull().sum() output?

import pandas as pd
data = {'A': [1, 2, None], 'B': [None, 2, 3], 'C': [1, 2, 3]}
df = pd.DataFrame(data)

medium

A. {'A': 1, 'B': 2, 'C': 1}

B. {'A': 0, 'B': 0, 'C': 0}

C. {'A': 2, 'B': 1, 'C': 0}

D. {'A': 1, 'B': 1, 'C': 0}

Solution

1. Step 1: Identify missing values in each column

   Column 'A' has one None (missing), 'B' has one None, and 'C' has no missing values.

2. Step 2: Understand what isnull().sum() does

   isnull() marks missing as True, and sum() counts True values per column.

3. Final Answer:

   {'A': 1, 'B': 1, 'C': 0} -> Option D

4. Quick Check:

   Count missing per column = isnull().sum() [OK]

Hint: Sum isnull() to count missing per column [OK]

Common Mistakes:

• Counting total missing instead of per column
• Confusing None with zero
• Misreading DataFrame values

4. The following code is intended to find duplicate rows in df. What is the error?

duplicates = df.duplicate()

medium

A. The method name should be duplicated(), not duplicate().

B. The DataFrame df is not defined.

C. The method duplicated() requires an argument.

D. The code should use df.isnull() to find duplicates.

Solution

1. Step 1: Check method name correctness

   The correct pandas method to find duplicates is duplicated(), not duplicate().

2. Step 2: Verify other options

   df is assumed defined, duplicated() does not require arguments, and isnull() checks missing data, not duplicates.

3. Final Answer:

   The method name should be duplicated(), not duplicate(). -> Option A

4. Quick Check:

   Correct method name = duplicated() [OK]

Hint: Remember: duplicated() has a 'd' at the end [OK]

Common Mistakes:

• Misspelling duplicated() as duplicate()
• Confusing duplicated() with isnull()
• Thinking duplicated() needs arguments

5. You have a DataFrame df with columns ['id', 'age', 'score']. You want to validate that age has no missing values and score has no duplicates. Which code correctly performs both checks?

hard

A. missing_age = df.isnull().sum()['age']
duplicate_score = df['score'].isnull().any()

B. missing_age = df['age'].isnull().sum()
duplicate_score = df['score'].duplicated().any()

C. missing_age = df['age'].duplicated().sum()
duplicate_score = df['score'].duplicated().sum()

D. missing_age = df['age'].duplicated().any()
duplicate_score = df['score'].isnull().sum()

Solution

1. Step 1: Check missing values in 'age'

   df['age'].isnull().sum() counts missing values correctly.

   missing_age = df['age'].isnull().sum()
   duplicate_score = df['score'].duplicated().any()

   uses this.

2. Step 2: Check duplicates in 'score'

   df['score'].duplicated().any() returns True if any duplicates exist, which is a good validation check.

3. Step 3: Review other options

   missing_age = df.isnull().sum()['age']
   duplicate_score = df['score'].isnull().any()

   wrongly checks duplicates with isnull(),

   missing_age = df['age'].duplicated().sum()
   duplicate_score = df['score'].duplicated().sum()

   uses duplicated() on 'age' which is wrong,

   missing_age = df['age'].duplicated().any()
   duplicate_score = df['score'].isnull().sum()

   wrongly uses duplicated() for missing and isnull() for duplicates.

4. Final Answer:

   missing_age = df['age'].isnull().sum(); duplicate_score = df['score'].duplicated().any() -> Option B

5. Quick Check:

   Missing count + duplicate check = correct validation [OK]

Hint: Sum isnull() for missing; use duplicated().any() for duplicates [OK]

Common Mistakes:

• Using duplicated() to count missing values
• Using isnull() to check duplicates
• Mixing sum() and any() incorrectly