Percentiles and quantiles in SciPy - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is a percentile in data analysis?

A percentile is a value below which a certain percentage of data falls. For example, the 25th percentile is the value below which 25% of the data lies.

beginner

How do quantiles relate to percentiles?

Quantiles divide data into equal-sized intervals. Percentiles are a type of quantile that divide data into 100 equal parts.

intermediate

Which function in SciPy can you use to calculate percentiles?

You can use scipy.stats.scoreatpercentile or numpy.percentile to calculate percentiles in Python.

beginner

What does the 50th percentile represent?

The 50th percentile is the median of the data. It splits the data into two equal halves.

intermediate

How can quantiles help in understanding data distribution?

Quantiles help by dividing data into equal parts, making it easier to see how data is spread and where values cluster or spread out.

What does the 90th percentile tell you about a dataset?

A 90% of data values are above this value

B It is the average of the top 10% values

C 90% of data values are below this value

D It is the maximum value in the dataset

The 90th percentile is the value below which 90% of the data falls.

Which Python library provides the function scoreatpercentile?

A scipy.stats

B matplotlib

C pandas

D seaborn

scoreatpercentile is part of the scipy.stats module.

If you want to split data into four equal parts, which quantiles do you use?

A Quartiles

B Percentiles

C Deciles

D Median

Quartiles split data into four equal parts.

What is the 0th percentile of a dataset?

A Maximum value

B Minimum value

C Median

D Mean

The 0th percentile is the minimum value in the dataset.

Which function can you use in NumPy to calculate the 75th percentile?

A numpy.quantile(data, 0.75)

B numpy.percentile(data, 75)

C numpy.mean(data)

D Both A and B

Both numpy.quantile(data, 0.75) and numpy.percentile(data, 75) calculate the 75th percentile.

Explain what percentiles and quantiles are and how they help in understanding data.

Think about how data is split into parts and what those parts tell us.

Describe how to calculate the 25th percentile of a dataset using SciPy.

Focus on the function and its parameters.

Practice

1. What does the 90th percentile represent in a dataset?

easy

A. The average of the top 90% of data

B. The value below which 90% of the data falls

C. The value above which 90% of the data falls

D. The middle value of the dataset

Solution

1. Step 1: Understand percentile definition

   A percentile indicates the value below which a given percentage of observations fall.

2. Step 2: Apply to 90th percentile

   The 90th percentile means 90% of data points are below this value.

3. Final Answer:

   The value below which 90% of the data falls -> Option B

4. Quick Check:

   Percentile = value below % data [OK]

Hint: Percentile = value below that percent of data [OK]

Common Mistakes:

• Confusing percentile with average
• Thinking percentile is value above percentage
• Mixing percentile with median

2. Which of the following is the correct way to find the 25th percentile using scipy.stats?

easy

A. stats.quantile(data, 25)

B. stats.percentile(data, 0.25)

C. stats.mquantiles(data, prob=0.25)

D. stats.scoreatpercentile(data, 25)

Solution

1. Step 1: Recall scipy function for percentile

   The function to find percentile is stats.scoreatpercentile with arguments (data, percentile).

2. Step 2: Check options

   stats.scoreatpercentile(data, 25) uses stats.scoreatpercentile(data, 25), which is correct for 25th percentile.

3. Final Answer:

   stats.scoreatpercentile(data, 25) -> Option D

4. Quick Check:

   Use scoreatpercentile for percentiles [OK]

Hint: Use stats.scoreatpercentile(data, percentile) [OK]

Common Mistakes:

• Using wrong function names like percentile or quantile
• Passing 0.25 instead of 25 for percentile
• Confusing quantiles with percentiles

3. Given the data array [10, 20, 30, 40, 50], what is the output of stats.scoreatpercentile(data, 40)?

medium

A. 26.0

B. 22.0

C. 30.0

D. 20.0

Solution

1. Step 1: Understand percentile calculation

   40th percentile means the value below which 40% of data lies. Data sorted: [10, 20, 30, 40, 50].

2. Step 2: Calculate 40th percentile using interpolation

   Position = (40/100)*(n-1) = 0.4*4 = 1.6. Interpolate between data[1]=20 and data[2]=30: 20 + 0.6*(30-20) = 20 + 6 = 26.0.

3. Final Answer:

   26.0 -> Option A

4. Quick Check:

   Interpolation gives 26.0 [OK]

Hint: Percentile position = p/100*(n-1), interpolate values [OK]

Common Mistakes:

• Choosing nearest data point without interpolation
• Using index p instead of p/100*(n-1)
• Confusing percentile with median

4. The code below is intended to find the quartiles of data using scipy but raises an error. What is the error?

import scipy.stats as stats
data = [5, 10, 15, 20, 25]
quartiles = stats.mstats.mquantiles(data, prob=[0.25, 0.5, 0.75])
print(quartiles)

medium

A. mquantiles requires numpy array, not list

B. prob argument should be a single float, not a list

C. No error, code runs correctly

D. mquantiles is not in scipy.stats.mstats

Solution

1. Step 1: Check function location and usage

   stats.mstats.mquantiles is a valid function and accepts prob as a list for multiple quantiles.

2. Step 2: Check input data type

   mquantiles accepts list or array-like data, so list input is valid.

3. Final Answer:

   No error, code runs correctly -> Option C

4. Quick Check:

   mquantiles accepts list and prob list [OK]

Hint: mquantiles accepts list and prob list [OK]

Common Mistakes:

• Assuming mquantiles only accepts numpy arrays
• Thinking prob must be a single float
• Confusing function location in scipy

5. You have a dataset of exam scores: [55, 70, 65, 80, 90, 85, 75]. You want to divide the scores into four equal groups (quartiles) using scipy. Which code snippet correctly computes the quartiles?

hard

A. stats.mstats.mquantiles(scores, prob=[0.25, 0.5, 0.75])

B. stats.scoreatpercentile(scores, [25, 50, 75])

C. np.percentile(scores, [0.25, 0.5, 0.75])

D. stats.mstats.mquantiles(scores, prob=[25, 50, 75])

Solution

1. Step 1: Identify correct function for quartiles

   Quartiles are quantiles at 25%, 50%, and 75%. stats.mstats.mquantiles accepts prob as fractions between 0 and 1.

2. Step 2: Check options for correct prob format

   stats.mstats.mquantiles(scores, prob=[0.25, 0.5, 0.75]) uses prob=[0.25, 0.5, 0.75], which is correct. stats.scoreatpercentile(scores, [25, 50, 75]) uses scoreatpercentile but with a list, which is invalid. np.percentile(scores, [0.25, 0.5, 0.75]) uses numpy but with fractions instead of percentages. stats.mstats.mquantiles(scores, prob=[25, 50, 75]) uses prob with values >1, which is invalid.

3. Final Answer:

   stats.mstats.mquantiles(scores, prob=[0.25, 0.5, 0.75]) -> Option A

4. Quick Check:

   mquantiles prob values are fractions [OK]

Hint: Use mquantiles with prob as fractions (0.25, 0.5, 0.75) [OK]

Common Mistakes:

• Passing percent values instead of fractions to prob
• Using scoreatpercentile with list of percentiles
• Confusing numpy percentile input format