Missing data strategies decision in Pandas - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Choosing the best strategy for missing data in a dataset

You have a dataset with 10% missing values in a column that represents age. The missing values are random and not related to other variables. Which missing data strategy is most appropriate?

A Fill missing age values with the mean age of the column.

B Remove all rows with missing age values.

C Fill missing age values with zero.

D Leave the missing values as they are.

💡 Hint

Think about preserving as much data as possible and using a simple method for random missing values.

Since missing values are random and only 10%, filling with the mean preserves data and avoids bias. Removing rows loses data unnecessarily. Filling with zero is misleading. Leaving missing values can cause errors in analysis.

❓ Predict Output

intermediate

Output of filling missing values with forward fill

What is the output of the following code?

Pandas

import pandas as pd

data = {'score': [10, None, None, 20, None, 30]}
df = pd.DataFrame(data)
df_filled = df.fillna(method='ffill')
print(df_filled)

A

   score
0   10.0
1   10.0
2   10.0
3   20.0
4   20.0
5    NaN

B

   score
0   10.0
1    NaN
2    NaN
3   20.0
4    NaN
5   30.0

C

   score
0   10.0
1   10.0
2   10.0
3   20.0
4    NaN
5   30.0

D

   score
0   10.0
1   10.0
2   10.0
3   20.0
4   20.0
5   30.0

💡 Hint

Forward fill replaces missing values with the last known value above.

Forward fill replaces each missing value with the previous non-missing value. So the None values after 10 become 10, after 20 become 20, and so on.

❓ data_output

advanced

Result of dropping rows with missing values

Given the DataFrame below, what is the result after dropping rows with any missing values?

Pandas

import pandas as pd

data = {'A': [1, 2, None, 4], 'B': [None, 2, 3, 4]}
df = pd.DataFrame(data)
df_clean = df.dropna()
print(df_clean)

A

   A    B
1  2.0  2.0
3  4.0  4.0

B Empty DataFrame with columns A and B and 0 rows

C

   A    B
0  1.0  NaN
1  2.0  2.0
2  NaN  3.0
3  4.0  4.0

D

   A    B
0  1.0  NaN
3  4.0  4.0

💡 Hint

dropna() removes rows with any missing value.

Rows 0 and 2 have missing values, so they are dropped. Rows 1 and 3 have no missing values and remain.

🔧 Debug

advanced

Identify the error in filling missing values with median

What error does the following code raise?

Pandas

import pandas as pd

data = {'height': [150, 160, None, 170, None]}
df = pd.DataFrame(data)
df['height'] = df['height'].fillna(df['height'].median())
print(df)

A TypeError because median() returns a string.

B No error, prints DataFrame with missing values filled by median.

C KeyError because 'height' column does not exist.

D AttributeError because fillna() is not a method of Series.

💡 Hint

Check if the methods and column names are correct.

The code correctly fills missing values with the median of the 'height' column. No error occurs.

🚀 Application

expert

Choosing the best missing data strategy for time series data

You have a time series dataset with missing temperature readings at random times. You want to fill missing values to keep the time order intact and avoid introducing bias. Which strategy is best?

A Fill missing values with the overall mean temperature.

B Remove all rows with missing temperature values.

C Use forward fill to propagate last known temperature forward.

D Fill missing values with zero.

💡 Hint

Think about preserving time order and realistic values.

Forward fill keeps the time order and uses the last known value, which is common for time series. Mean or zero can distort trends. Removing rows loses data and breaks continuity.

Practice

1.

What is the main purpose of using dropna() in pandas?

easy

A. To remove rows or columns that contain missing data

B. To replace missing data with zeros

C. To fill missing data with the mean value

D. To convert data types of columns

Solution

1. Step 1: Understand dropna() function

   dropna() removes rows or columns that have missing values (NaN).

2. Step 2: Compare with other options

   Replacing missing data or converting data types are done by other functions like fillna() or astype().

3. Final Answer:

   To remove rows or columns that contain missing data -> Option A

4. Quick Check:

   dropna() removes missing data rows/columns [OK]

Hint: Remember: dropna drops missing data, fillna fills it [OK]

Common Mistakes:

• Confusing dropna with fillna
• Thinking dropna fills missing values
• Assuming dropna changes data types

2.

Which of the following is the correct syntax to fill missing values in a DataFrame df with the value 0?

?

easy

A. df.fillna(0)

B. df.dropna(0)

C. df.fill(na=0)

D. df.replace(na=0)

Solution

1. Step 1: Recall fillna syntax

   The correct method to fill missing values is fillna() with the value as argument.

2. Step 2: Check each option

   df.fillna(0) uses df.fillna(0) which is correct. Others have wrong method names or parameters.

3. Final Answer:

   df.fillna(0) -> Option A

4. Quick Check:

   Use fillna(value) to fill missing values [OK]

Hint: fillna fills missing values; dropna removes rows/columns [OK]

Common Mistakes:

• Using dropna instead of fillna
• Wrong method name like fill() or replace()
• Passing parameters incorrectly

3.

Given the DataFrame df below, what will be the output of df.fillna(method='ffill')?

import pandas as pd
df = pd.DataFrame({'A': [1, None, 3, None], 'B': [None, 2, None, 4]})
print(df.fillna(method='ffill'))

medium

A. Removes rows with missing values

B. Fills missing values with zero

C. Fills missing values by propagating last valid observation forward

D. Raises an error due to wrong method

Solution

1. Step 1: Understand forward fill method

   method='ffill' fills missing values by copying the last known non-missing value forward.

2. Step 2: Apply to DataFrame

   For column A: 1 stays, None becomes 1, 3 stays, None becomes 3. For B: None stays None (no previous), 2 stays, None becomes 2, 4 stays.

3. Final Answer:

   Fills missing values by propagating last valid observation forward -> Option C

4. Quick Check:

   ffill copies last valid value forward [OK]

Hint: ffill copies previous value forward to fill NaNs [OK]

Common Mistakes:

• Thinking fillna(method='ffill') fills with zero
• Confusing ffill with dropping rows
• Expecting error from method parameter

4.

What is wrong with the following code if the goal is to remove all rows with missing data from DataFrame df?

df.fillna(drop=True)

medium

A. The code should use df.dropna(inplace=True) to modify in place

B. fillna() does not have a drop parameter; use dropna() instead

C. The parameter drop=True should be dropna=True

D. The code is correct and will remove rows with missing data

Solution

1. Step 1: Check method and parameters

   fillna() is for filling missing values and does not accept a drop parameter.

2. Step 2: Correct method for removing rows

   To remove rows with missing data, use dropna() instead.

3. Final Answer:

   fillna() does not have a drop parameter; use dropna() instead -> Option B

4. Quick Check:

   Use dropna() to remove missing data rows [OK]

Hint: Use dropna() to remove rows, fillna() to replace values [OK]

Common Mistakes:

• Using fillna() with invalid parameters
• Confusing fillna and dropna functions
• Expecting fillna to remove rows

5.

You have a DataFrame df with many missing values scattered randomly. You want to keep as much data as possible but avoid rows with more than 2 missing values. Which pandas method and parameter combination is best?

hard

A. Use df.fillna(method='bfill') to fill missing values backward

B. Use df.fillna(0) to replace all missing values with zero

C. Use df.dropna(axis=1) to drop columns with missing values

D. Use df.dropna(thresh=len(df.columns)-2) to drop rows with more than 2 missing values

Solution

1. Step 1: Understand the threshold parameter

   dropna(thresh=n) keeps rows with at least n non-missing values.

2. Step 2: Calculate threshold for max 2 missing values

   If total columns are len(df.columns), then rows must have at least len(df.columns) - 2 non-missing values to keep.

3. Step 3: Evaluate other options

   Filling with zero or backward fill changes data but does not drop rows. Dropping columns removes data unnecessarily.

4. Final Answer:

   Use df.dropna(thresh=len(df.columns)-2) to drop rows with more than 2 missing values -> Option D

5. Quick Check:

   dropna(thresh) keeps rows with enough non-NaNs [OK]

Hint: Use dropna(thresh) to keep rows with few missing values [OK]

Common Mistakes:

• Using fillna instead of dropping rows
• Dropping columns instead of rows
• Misunderstanding thresh parameter