Percentiles with np.percentile() in NumPy

Introduction

Percentiles help us understand the position of a value in a list compared to others. Using np.percentile() lets us find these positions easily.

To find the value below which a certain percentage of data falls, like the 25th percentile (first quartile).

To understand the spread of exam scores and see what score separates the top 10% from the rest.

To detect outliers by checking values at very low or very high percentiles.

To summarize data distribution without looking at every number.

Syntax

NumPy

np.percentile(data, percentile, axis=None, interpolation='linear')

data is your list or array of numbers.

percentile is a number between 0 and 100 showing which percentile to find.

Examples

Finds the 50th percentile (median) of the list.

NumPy

np.percentile([10, 20, 30, 40, 50], 50)

Finds the 25th percentile (first quartile) of the list.

NumPy

np.percentile([1, 2, 3, 4, 5], 25)

Finds the 50th percentile for each column in a 2D array.

NumPy

np.percentile([[10, 20], [30, 40]], 50, axis=0)

Sample Program

This code finds key percentiles of exam scores to understand their distribution.

NumPy

import numpy as np

# Sample data: exam scores
scores = [55, 70, 65, 90, 85, 75, 60]

# Find the 25th, 50th, and 75th percentiles
p25 = np.percentile(scores, 25)
p50 = np.percentile(scores, 50)
p75 = np.percentile(scores, 75)

print(f"25th percentile: {p25}")
print(f"50th percentile (median): {p50}")
print(f"75th percentile: {p75}")

Important Notes

Percentiles split data into 100 equal parts.

If the exact percentile position is between two data points, np.percentile() interpolates by default.

Summary

Percentiles show the relative standing of values in data.

np.percentile() is an easy way to calculate them in Python.

Use percentiles to summarize data and find thresholds.

Practice

1. What does the function np.percentile() calculate in a dataset?

easy

A. The value below which a given percentage of data falls

B. The average value of the dataset

C. The total sum of all data points

D. The number of data points in the dataset

Solution

1. Step 1: Understand percentile meaning

   A percentile indicates the value below which a certain percentage of data lies.

2. Step 2: Connect to np.percentile()

   The function np.percentile() returns this value for the given percentage.

3. Final Answer:

   The value below which a given percentage of data falls -> Option A

4. Quick Check:

   Percentile = value below % data [OK]

Hint: Percentile = cutoff value for given percent [OK]

Common Mistakes:

• Confusing percentile with mean or sum
• Thinking percentile counts data points
• Mixing percentile with total dataset size

2. Which of the following is the correct syntax to find the 25th percentile of a numpy array data?

easy

A. np.percentile(25, data)

B. np.percentile(data, 25)

C. np.percentile(data, percentile=25)

D. np.percentile(25%)

Solution

1. Step 1: Recall np.percentile() parameters

   The first argument is the data array, the second is the percentile number.

2. Step 2: Check each option

   np.percentile(data, 25) correctly uses np.percentile(data, 25). Others have wrong argument order or syntax.

3. Final Answer:

   np.percentile(data, 25) -> Option B

4. Quick Check:

   np.percentile(array, percentile) [OK]

Hint: First argument data, second percentile number [OK]

Common Mistakes:

• Swapping arguments order
• Using keyword 'percentile' incorrectly
• Adding % sign inside function

3. Given the array arr = np.array([10, 20, 30, 40, 50]), what is the output of np.percentile(arr, 40)?

medium

A. 20.0

B. 30.0

C. 26.0

D. 40.0

Solution

1. Step 1: Understand percentile calculation

   40th percentile means value below which 40% of data lies. Data sorted: [10,20,30,40,50]

2. Step 2: Calculate 40th percentile

   Position = (40/100)*(5-1) = 1.6. Interpolate between arr[1]=20 and arr[2]=30: 20 + 0.6*(30-20) = 26.0

3. Final Answer:

   26.0 -> Option C

4. Quick Check:

   40th percentile = 26.0 [OK]

Hint: Percentile index = p/100*(n-1), interpolate if needed [OK]

Common Mistakes:

• Picking nearest data point without interpolation
• Using wrong index calculation
• Confusing percentile with quartile

4. What is wrong with this code snippet to find the 90th percentile of data?

np.percentile(data, percentile=90)

medium

A. The data array must be sorted before calling np.percentile()

B. The percentile value must be between 0 and 1, not 90

C. The function should be called as data.percentile(90)

D. The keyword argument 'percentile' is invalid; percentile should be positional

Solution

1. Step 1: Check np.percentile() argument rules

   The function expects the percentile as the second positional argument, not as a keyword 'percentile'.

2. Step 2: Validate other options

   Percentile can be 90 (not 0.9), function is not a method of array, and sorting is handled internally.

3. Final Answer:

   The keyword argument 'percentile' is invalid; percentile should be positional -> Option D

4. Quick Check:

   Percentile argument is positional [OK]

Hint: Percentile must be second positional argument [OK]

Common Mistakes:

• Using invalid keyword argument
• Assuming percentile is fraction (0-1)
• Trying to sort data manually

5. You have a dataset data = np.array([5, 7, 8, 12, 15, 18, 22, 27, 30]). You want to find the value below which 75% of the data falls and then filter the dataset to only include values below this percentile. Which code correctly does this?

hard

A. threshold = np.percentile(data, 75)
filtered = data[data <= threshold]

B. threshold = np.percentile(data, 0.75)
filtered = data[data <= threshold]

C. threshold = np.percentile(data, 75)
filtered = data[data < threshold]

D. threshold = np.percentile(data, 75)
filtered = data[data > threshold]

Solution

1. Step 1: Calculate 75th percentile threshold

   Use np.percentile(data, 75) to get the value below which 75% data lies.

2. Step 2: Filter data including values equal to threshold

   Use <= to include values equal to threshold, so filtered = data[data <= threshold].

3. Final Answer:

   threshold = np.percentile(data, 75) filtered = data[data <= threshold] -> Option A

4. Quick Check:

   Percentile 75 + filter <= threshold [OK]

Hint: Use percentile value and filter with <= threshold [OK]

Common Mistakes:

• Using 0.75 instead of 75 for percentile
• Filtering with < instead of <=
• Filtering with > instead of <= or <