Outlier detection with IQR in Pandas - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

What is the output of this IQR outlier detection code?

Given the DataFrame df below, what does the outliers variable contain after running the code?

Pandas

import pandas as pd

df = pd.DataFrame({'values': [10, 12, 14, 15, 18, 19, 20, 100]})
Q1 = df['values'].quantile(0.25)
Q3 = df['values'].quantile(0.75)
IQR = Q3 - Q1
outliers = df[(df['values'] < Q1 - 1.5 * IQR) | (df['values'] > Q3 + 1.5 * IQR)]
print(outliers)

A

   values
7     100

B Empty DataFrame\nColumns: [values]\nIndex: []

C

   values
0      10
7    100

D

   values
0     10

💡 Hint

Calculate Q1 and Q3, then find the IQR. Check which values fall outside the range Q1 - 1.5*IQR to Q3 + 1.5*IQR.

The IQR is Q3 - Q1 = 19 - 12 = 7. The lower bound is 12 - 1.5*7 = 1.5 and the upper bound is 19 + 1.5*7 = 29.5. Only the value 100 is above 29.5, so it is the only outlier.

❓ data_output

intermediate

How many outliers are detected by this IQR method?

Using the DataFrame df and the IQR method below, how many rows are identified as outliers?

Pandas

import pandas as pd

df = pd.DataFrame({'scores': [55, 60, 65, 70, 75, 80, 85, 90, 95, 200]})
Q1 = df['scores'].quantile(0.25)
Q3 = df['scores'].quantile(0.75)
IQR = Q3 - Q1
outliers = df[(df['scores'] < Q1 - 1.5 * IQR) | (df['scores'] > Q3 + 1.5 * IQR)]
print(len(outliers))

A 2

B 0

C 1

D 3

💡 Hint

Calculate the IQR and check which values fall outside the bounds.

Q1 is 67.5, Q3 is 87.5, so IQR is 20. Lower bound is 67.5 - 1.5*20 = 37.5, upper bound is 87.5 + 1.5*20 = 117.5. Only 200 is outside this range, so 1 outlier.

❓ visualization

advanced

Which boxplot correctly shows the outliers detected by IQR?

You run this code to detect outliers and plot a boxplot. Which option shows the correct boxplot visualization?

Pandas

import pandas as pd
import matplotlib.pyplot as plt

df = pd.DataFrame({'data': [5, 7, 8, 9, 10, 12, 15, 100]})
plt.boxplot(df['data'])
plt.show()

A Boxplot with one point far below the lower whisker at 5

B Boxplot with one point far above the upper whisker at 100

C Boxplot with multiple points above whiskers at 15 and 100

D Boxplot with no points outside whiskers

💡 Hint

Outliers appear as points outside the whiskers. The value 100 is much larger than others.

The boxplot shows one outlier point at 100 above the upper whisker. The lower whisker includes 5, so no outlier below.

🔧 Debug

advanced

What error does this IQR outlier detection code raise?

Identify the error raised by this code snippet:

Pandas

import pandas as pd

df = pd.DataFrame({'vals': [1, 2, 3, 4, 5]})
Q1 = df['vals'].quantile(0.25)
Q3 = df['vals'].quantile(0.75)
IQR = Q3 - Q1
outliers = df[(df['vals'] < Q1 - 1.5 * IQR) | (df['vals'] > Q3 + 1.5 * IQR)]
print(outlier)

A SyntaxError: invalid syntax

B TypeError: unsupported operand type(s) for -: 'str' and 'float'

C KeyError: 'vals'

D NameError: name 'outlier' is not defined

💡 Hint

Check variable names carefully in the print statement.

The variable is named 'outliers' but the code tries to print 'outlier', which is undefined.

🚀 Application

expert

Which option correctly filters out outliers using IQR in a DataFrame with multiple columns?

You have a DataFrame df with columns A and B. You want to remove rows where A or B have outliers based on IQR. Which code correctly does this?

Pandas

import pandas as pd

df = pd.DataFrame({'A': [10, 12, 14, 100, 15], 'B': [20, 22, 23, 24, 200]})

A

Q1 = df.quantile(0.25)
Q3 = df.quantile(0.75)
IQR = Q3 - Q1
filtered = df[(df['A'] >= Q1['A'] - 1.5 * IQR['A']) & (df['A'] <= Q3['A'] + 1.5 * IQR['A']) & (df['B'] >= Q1['B'] - 1.5 * IQR['B']) & (df['B'] <= Q3['B'] + 1.5 * IQR['B'])]

B

Q1 = df.quantile(0.25)
Q3 = df.quantile(0.75)
IQR = Q3 - Q1
filtered = df[(df >= Q1 - 1.5 * IQR) & (df <= Q3 + 1.5 * IQR)]

C

Q1_A = df['A'].quantile(0.25)
Q3_A = df['A'].quantile(0.75)
IQR_A = Q3_A - Q1_A
Q1_B = df['B'].quantile(0.25)
Q3_B = df['B'].quantile(0.75)
IQR_B = Q3_B - Q1_B
filtered = df[~((df['A'] < Q1_A - 1.5 * IQR_A) | (df['A'] > Q3_A + 1.5 * IQR_A) | (df['B'] < Q1_B - 1.5 * IQR_B) | (df['B'] > Q3_B + 1.5 * IQR_B))]

D

Q1 = df.quantile(0.25)
Q3 = df.quantile(0.75)
IQR = Q3 - Q1
filtered = df[(df['A'] >= Q1['A'] - 1.5 * IQR['A']) | (df['A'] <= Q3['A'] + 1.5 * IQR['A']) | (df['B'] >= Q1['B'] - 1.5 * IQR['B']) | (df['B'] <= Q3['B'] + 1.5 * IQR['B'])]

💡 Hint

Use logical AND to keep rows where both columns are within the IQR bounds.

Option A correctly applies the IQR filter for each column using AND conditions to keep rows where neither column is an outlier. Option A uses negation but is correct logically; however, it is verbose. Option A uses DataFrame-wide comparison which returns NaNs and does not filter properly. Option A uses OR with >= and <= which includes all rows and is incorrect.

Practice

1. What does the Interquartile Range (IQR) represent in a dataset?

easy

A. The range between the 25th percentile (Q1) and the 75th percentile (Q3)

B. The difference between the maximum and minimum values

C. The average value of the dataset

D. The total number of data points

Solution

1. Step 1: Understand quartiles

   Quartiles divide data into four equal parts. Q1 is the 25th percentile, Q3 is the 75th percentile.

2. Step 2: Define IQR

   IQR is the difference between Q3 and Q1, showing the middle 50% spread of data.

3. Final Answer:

   The range between the 25th percentile (Q1) and the 75th percentile (Q3) -> Option A

4. Quick Check:

   IQR = Q3 - Q1 [OK]

Hint: IQR = Q3 minus Q1, middle half range [OK]

Common Mistakes:

• Confusing IQR with total range
• Thinking IQR is the average
• Mixing up quartiles with percentiles

2. Which of the following pandas code snippets correctly calculates the IQR of a DataFrame column df['score']?

easy

A. iqr = df['score'].mean() - df['score'].median()

B. iqr = df['score'].max() - df['score'].min()

C. iqr = df['score'].quantile(0.75) - df['score'].quantile(0.25)

D. iqr = df['score'].std()

Solution

1. Step 1: Recall IQR formula

   IQR is Q3 minus Q1, where Q3 is 75th percentile and Q1 is 25th percentile.

2. Step 2: Check pandas quantile usage

   Using quantile(0.75) and quantile(0.25) correctly gets Q3 and Q1.

3. Final Answer:

   iqr = df['score'].quantile(0.75) - df['score'].quantile(0.25) -> Option C

4. Quick Check:

   Use quantile(0.75) - quantile(0.25) for IQR [OK]

Hint: Use quantile(0.75) minus quantile(0.25) for IQR [OK]

Common Mistakes:

• Using mean or median instead of quantiles
• Using max-min instead of IQR
• Using standard deviation instead of IQR

3. Given the following code, what will be the output of outliers?

import pandas as pd

data = {'values': [10, 12, 14, 15, 18, 100, 105]}
df = pd.DataFrame(data)
Q1 = df['values'].quantile(0.25)
Q3 = df['values'].quantile(0.75)
IQR = Q3 - Q1
lower_bound = Q1 - 1.5 * IQR
upper_bound = Q3 + 1.5 * IQR
outliers = df[(df['values'] < lower_bound) | (df['values'] > upper_bound)]
print(outliers)

medium

A. Rows with values 10 and 12

B. Rows with values 100 and 105

C. All rows because 1.5*IQR is too small

D. Empty DataFrame, no outliers detected

Solution

1. Step 1: Calculate Q1, Q3, and IQR

   Q1 = 12.0 (between 12 and 14), Q3 = 18.0 (between 18 and 100), IQR = 6.0.

2. Step 2: Calculate bounds and find outliers

   Lower bound = 12 - 1.5*6 = 3; Upper bound = 18 + 1.5*6 = 27. Values 100 and 105 are above upper bound, so they are outliers.

3. Final Answer:

   Rows with values 100 and 105 -> Option B

4. Quick Check:

   Outliers > Q3 + 1.5*IQR or < Q1 - 1.5*IQR: 100, 105 [OK]

Hint: Outliers lie beyond Q1-1.5*IQR or Q3+1.5*IQR [OK]

Common Mistakes:

• Miscomputing Q1 or Q3
• Using wrong inequality signs
• Ignoring both lower and upper bounds

4. Identify the error in this code snippet for detecting outliers using IQR:

Q1 = df['data'].quantile(0.25)
Q3 = df['data'].quantile(0.75)
IQR = Q3 - Q1
outliers = df[(df['data'] > Q3 + 1.5 * IQR) & (df['data'] < Q1 - 1.5 * IQR)]

medium

A. No error, code is correct

B. IQR calculation is incorrect

C. Quantile values are swapped

D. The condition uses '&' instead of '|' to combine bounds

Solution

1. Step 1: Understand outlier condition

   Outliers are values less than lower bound OR greater than upper bound, so conditions should be combined with OR (|).

2. Step 2: Check code condition

   The code uses AND (&), which means no value can be both greater than upper bound and less than lower bound simultaneously.

3. Final Answer:

   The condition uses '&' instead of '|' to combine bounds -> Option D

4. Quick Check:

   Use '|' for outlier condition, not '&' [OK]

Hint: Use OR (|) to combine outlier conditions, not AND (&) [OK]

Common Mistakes:

• Using AND instead of OR for outlier filtering
• Swapping Q1 and Q3 values
• Calculating IQR incorrectly

5. You have a DataFrame df with a column 'height'. You want to remove outliers using IQR but keep the original index. Which code correctly filters out outliers and keeps the index intact?

hard

A. Q1 = df['height'].quantile(0.25)
Q3 = df['height'].quantile(0.75)
IQR = Q3 - Q1
filtered_df = df[(df['height'] >= Q1 - 1.5*IQR) & (df['height'] <= Q3 + 1.5*IQR)]

B. Q1 = df['height'].quantile(0.25)
Q3 = df['height'].quantile(0.75)
IQR = Q3 - Q1
filtered_df = df.reset_index(drop=True)[(df['height'] >= Q1 - 1.5*IQR) & (df['height'] <= Q3 + 1.5*IQR)]

C. Q1 = df['height'].quantile(0.25)
Q3 = df['height'].quantile(0.75)
IQR = Q3 - Q1
filtered_df = df[(df['height'] > Q1 + 1.5*IQR) | (df['height'] < Q3 - 1.5*IQR)]

D. Q1 = df['height'].quantile(0.25)
Q3 = df['height'].quantile(0.75)
IQR = Q3 - Q1
filtered_df = df[(df['height'] < Q1 - 1.5*IQR) & (df['height'] > Q3 + 1.5*IQR)]

Solution

1. Step 1: Calculate IQR and bounds correctly

   Calculate Q1, Q3, and IQR. Define lower bound as Q1 - 1.5*IQR and upper bound as Q3 + 1.5*IQR.

2. Step 2: Filter rows within bounds and keep index

   Use boolean indexing with AND (&) to keep rows inside bounds. Do not reset index to keep original indices.

3. Final Answer:

   Code in option A correctly filters and keeps index -> Option A

4. Quick Check:

   Filter with AND inside bounds, no reset_index [OK]

Hint: Filter with AND inside bounds; avoid resetting index [OK]

Common Mistakes:

• Using OR instead of AND to filter
• Resetting index and losing original row labels
• Swapping bounds or using wrong inequalities