Percentiles and quantiles in SciPy - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Output of percentile calculation with scipy

What is the output of this code snippet that calculates the 75th percentile of the data array?

SciPy

import numpy as np
from scipy import stats

data = np.array([10, 20, 30, 40, 50])
result = stats.scoreatpercentile(data, 75)
print(result)

A 40.0

B 45.0

C 42.5

D 37.5

💡 Hint

Remember that the 75th percentile is the value below which 75% of the data fall. scipy's scoreatpercentile uses linear interpolation by default.

The 75th percentile lies between 30 and 40. Using linear interpolation, it is 30 + 0.75*(40-30) = 37.5.

❓ data_output

intermediate

Number of elements below the 60th percentile

Given this dataset, how many elements are below the 60th percentile calculated using scipy?

SciPy

import numpy as np
from scipy import stats

data = np.array([5, 15, 25, 35, 45, 55, 65])
percentile_60 = stats.scoreatpercentile(data, 60)
count_below = np.sum(data < percentile_60)
print(count_below)

A 3

B 4

C 5

D 6

💡 Hint

Calculate the 60th percentile value first, then count how many data points are strictly less than that value.

The 60th percentile is 41.0. Data points less than 41.0 are 5, 15, 25, 35 (4 elements).

❓ visualization

advanced

Visualizing quantiles with a boxplot

Which option produces a boxplot showing the quartiles of the data array using matplotlib and scipy?

SciPy

import numpy as np
import matplotlib.pyplot as plt
from scipy import stats

data = np.array([12, 7, 3, 15, 8, 10, 6, 9, 11, 14])

q1 = stats.scoreatpercentile(data, 25)
q2 = stats.scoreatpercentile(data, 50)
q3 = stats.scoreatpercentile(data, 75)

plt.boxplot(data)
plt.title(f"Quartiles: Q1={q1}, Median={q2}, Q3={q3}")
plt.show()

A A boxplot with whiskers showing min and max, and a box from 7 to 12 with median at 9

B A boxplot with whiskers showing min and max, and a box from 6 to 15 with median at 10

C A boxplot with whiskers showing min and max, and a box from 7.25 to 11.75 with median at 9.5

D A boxplot with whiskers showing min and max, and a box from 3 to 15 with median at 9

💡 Hint

Check the quartile values calculated by scipy and compare with the boxplot box edges and median.

The 25th percentile (Q1) is 7.25, 50th percentile (median) is 9.5, and 75th percentile (Q3) is 11.75. The boxplot box edges correspond to Q1 and Q3, and the line inside is the median.

🧠 Conceptual

advanced

Understanding interpolation methods in percentile calculation

Which statement correctly describes the effect of changing the interpolation method in scipy's percentile calculation?

A Using 'nearest' interpolation always returns a data point from the dataset as the percentile value.

B Using 'higher' interpolation returns the smallest data point in the dataset.

C Using 'lower' interpolation averages the two closest data points around the percentile.

D Using 'linear' interpolation returns the closest data point without averaging.

💡 Hint

Think about how each interpolation method chooses the percentile value relative to the data points.

'Nearest' interpolation picks the closest actual data point to the percentile position. 'Linear' interpolates between points. 'Lower' picks the smaller data point at the percentile position. 'Higher' picks the larger data point.

🔧 Debug

expert

Identify the error in percentile calculation code

What error will this code raise when executed?

SciPy

import numpy as np
from scipy import stats

data = np.array([1, 2, 3, 4, 5])
result = stats.scoreatpercentile(data, 110)
print(result)

A ValueError: percentile must be in the range [0, 100]

B IndexError: index out of bounds

C TypeError: unsupported operand type(s)

D No error, outputs 5

💡 Hint

Percentiles must be between 0 and 100 inclusive.

Passing a percentile value greater than 100 causes a ValueError because percentiles are defined only between 0 and 100.

Practice

1. What does the 90th percentile represent in a dataset?

easy

A. The average of the top 90% of data

B. The value below which 90% of the data falls

C. The value above which 90% of the data falls

D. The middle value of the dataset

Solution

1. Step 1: Understand percentile definition

   A percentile indicates the value below which a given percentage of observations fall.

2. Step 2: Apply to 90th percentile

   The 90th percentile means 90% of data points are below this value.

3. Final Answer:

   The value below which 90% of the data falls -> Option B

4. Quick Check:

   Percentile = value below % data [OK]

Hint: Percentile = value below that percent of data [OK]

Common Mistakes:

• Confusing percentile with average
• Thinking percentile is value above percentage
• Mixing percentile with median

2. Which of the following is the correct way to find the 25th percentile using scipy.stats?

easy

A. stats.quantile(data, 25)

B. stats.percentile(data, 0.25)

C. stats.mquantiles(data, prob=0.25)

D. stats.scoreatpercentile(data, 25)

Solution

1. Step 1: Recall scipy function for percentile

   The function to find percentile is stats.scoreatpercentile with arguments (data, percentile).

2. Step 2: Check options

   stats.scoreatpercentile(data, 25) uses stats.scoreatpercentile(data, 25), which is correct for 25th percentile.

3. Final Answer:

   stats.scoreatpercentile(data, 25) -> Option D

4. Quick Check:

   Use scoreatpercentile for percentiles [OK]

Hint: Use stats.scoreatpercentile(data, percentile) [OK]

Common Mistakes:

• Using wrong function names like percentile or quantile
• Passing 0.25 instead of 25 for percentile
• Confusing quantiles with percentiles

3. Given the data array [10, 20, 30, 40, 50], what is the output of stats.scoreatpercentile(data, 40)?

medium

A. 26.0

B. 22.0

C. 30.0

D. 20.0

Solution

1. Step 1: Understand percentile calculation

   40th percentile means the value below which 40% of data lies. Data sorted: [10, 20, 30, 40, 50].

2. Step 2: Calculate 40th percentile using interpolation

   Position = (40/100)*(n-1) = 0.4*4 = 1.6. Interpolate between data[1]=20 and data[2]=30: 20 + 0.6*(30-20) = 20 + 6 = 26.0.

3. Final Answer:

   26.0 -> Option A

4. Quick Check:

   Interpolation gives 26.0 [OK]

Hint: Percentile position = p/100*(n-1), interpolate values [OK]

Common Mistakes:

• Choosing nearest data point without interpolation
• Using index p instead of p/100*(n-1)
• Confusing percentile with median

4. The code below is intended to find the quartiles of data using scipy but raises an error. What is the error?

import scipy.stats as stats
data = [5, 10, 15, 20, 25]
quartiles = stats.mstats.mquantiles(data, prob=[0.25, 0.5, 0.75])
print(quartiles)

medium

A. mquantiles requires numpy array, not list

B. prob argument should be a single float, not a list

C. No error, code runs correctly

D. mquantiles is not in scipy.stats.mstats

Solution

1. Step 1: Check function location and usage

   stats.mstats.mquantiles is a valid function and accepts prob as a list for multiple quantiles.

2. Step 2: Check input data type

   mquantiles accepts list or array-like data, so list input is valid.

3. Final Answer:

   No error, code runs correctly -> Option C

4. Quick Check:

   mquantiles accepts list and prob list [OK]

Hint: mquantiles accepts list and prob list [OK]

Common Mistakes:

• Assuming mquantiles only accepts numpy arrays
• Thinking prob must be a single float
• Confusing function location in scipy

5. You have a dataset of exam scores: [55, 70, 65, 80, 90, 85, 75]. You want to divide the scores into four equal groups (quartiles) using scipy. Which code snippet correctly computes the quartiles?

hard

A. stats.mstats.mquantiles(scores, prob=[0.25, 0.5, 0.75])

B. stats.scoreatpercentile(scores, [25, 50, 75])

C. np.percentile(scores, [0.25, 0.5, 0.75])

D. stats.mstats.mquantiles(scores, prob=[25, 50, 75])

Solution

1. Step 1: Identify correct function for quartiles

   Quartiles are quantiles at 25%, 50%, and 75%. stats.mstats.mquantiles accepts prob as fractions between 0 and 1.

2. Step 2: Check options for correct prob format

   stats.mstats.mquantiles(scores, prob=[0.25, 0.5, 0.75]) uses prob=[0.25, 0.5, 0.75], which is correct. stats.scoreatpercentile(scores, [25, 50, 75]) uses scoreatpercentile but with a list, which is invalid. np.percentile(scores, [0.25, 0.5, 0.75]) uses numpy but with fractions instead of percentages. stats.mstats.mquantiles(scores, prob=[25, 50, 75]) uses prob with values >1, which is invalid.

3. Final Answer:

   stats.mstats.mquantiles(scores, prob=[0.25, 0.5, 0.75]) -> Option A

4. Quick Check:

   mquantiles prob values are fractions [OK]

Hint: Use mquantiles with prob as fractions (0.25, 0.5, 0.75) [OK]

Common Mistakes:

• Passing percent values instead of fractions to prob
• Using scoreatpercentile with list of percentiles
• Confusing numpy percentile input format