Why systematic cleaning matters in Pandas - Challenge Your Understanding

Challenge - 5 Problems

❓ data_output

intermediate

Effect of Missing Data on Mean Calculation

Consider a dataset with missing values. What is the output of the mean calculation without cleaning?

Pandas

import pandas as pd

data = {'score': [10, 20, None, 40, 50]}
df = pd.DataFrame(data)
mean_score = df['score'].mean()
print(mean_score)

A TypeError

B None

C SyntaxError

D 30.0

💡 Hint

Think about how pandas handles missing values in calculations.

Pandas automatically ignores None or NaN values when calculating the mean, so the mean is calculated over the available numbers only.

❓ data_output

intermediate

Impact of Duplicate Rows on Data Analysis

Given a DataFrame with duplicate rows, what is the number of unique rows after removing duplicates?

Pandas

import pandas as pd

data = {'id': [1, 2, 2, 3, 4, 4, 4], 'value': [10, 20, 20, 30, 40, 40, 40]}
df = pd.DataFrame(data)
unique_rows = df.drop_duplicates().shape[0]
print(unique_rows)

A 4

B 5

C 3

D 7

💡 Hint

Removing duplicates keeps only one instance of each repeated row.

Rows with ids 2 and 4 are duplicated. After dropping duplicates, only unique rows remain, totaling 4.

🧠 Conceptual

advanced

Why Systematic Cleaning Prevents Analysis Errors

Which statement best explains why systematic data cleaning is important before analysis?

A It helps identify and fix inconsistencies that could lead to wrong conclusions.

B It ensures all data is removed to avoid errors.

C It speeds up the analysis by deleting half the data randomly.

D It replaces all numbers with zeros to simplify calculations.

💡 Hint

Think about the role of data quality in making good decisions.

Systematic cleaning finds and fixes problems like missing or wrong data, so the analysis results are trustworthy.

❓ Predict Output

advanced

Result of Cleaning and Filtering Data

What is the output DataFrame after cleaning missing values and filtering scores above 25?

Pandas

import pandas as pd

data = {'name': ['Anna', 'Bob', 'Cara', 'Dan'], 'score': [20, None, 30, 40]}
df = pd.DataFrame(data)
df_clean = df.dropna()
df_filtered = df_clean[df_clean['score'] > 25]
print(df_filtered)

A

   name  score
0  Anna   20.0
2  Cara   30.0
3   Dan   40.0

B

   name  score
2  Cara   30.0
3   Dan   40.0

C

   name  score
1   Bob    NaN
2  Cara   30.0
3   Dan   40.0

D

Empty DataFrame
Columns: [name, score]
Index: []

💡 Hint

First remove rows with missing scores, then keep only scores above 25.

Row with Bob is removed due to missing score. Anna's score is 20, below 25, so only Cara and Dan remain.

🚀 Application

expert

Identify the Impact of Unclean Data on Grouped Aggregation

Given this dataset, what is the average score per group if missing values are NOT cleaned?

Pandas

import pandas as pd
import numpy as np

data = {'group': ['A', 'A', 'B', 'B', 'C', 'C'], 'score': [10, np.nan, 20, 30, np.nan, 50]}
df = pd.DataFrame(data)
grouped_mean = df.groupby('group')['score'].mean()
print(grouped_mean)

A

group
A    NaN
B    25.0
C    NaN
Name: score, dtype: float64

B

group
A    10.0
B    25.0
C    0.0
Name: score, dtype: float64

C

group
A    10.0
B    25.0
C    50.0
Name: score, dtype: float64

D

group
A    20.0
B    25.0
C    50.0
Name: score, dtype: float64

💡 Hint

Remember how pandas handles missing values in group calculations.

Pandas ignores missing values in mean calculations per group. Group A has one valid score 10, group B has 20 and 30 averaging 25, group C has one valid score 50.