Pearson correlation in SciPy - Mini Project: Build & Apply

Calculate Pearson Correlation with SciPy

📖 Scenario: You are analyzing how two sets of exam scores relate to each other. You want to find out if students who score high in one exam also score high in the other.

🎯 Goal: Calculate the Pearson correlation coefficient between two lists of exam scores using the scipy.stats library.

📋 What You'll Learn

Create two lists called exam1_scores and exam2_scores with exact values

Import the pearsonr function from scipy.stats

Use pearsonr to calculate the correlation coefficient between the two lists

Print the Pearson correlation coefficient value

💡 Why This Matters

🌍 Real World

Pearson correlation is used in many fields like education, health, and business to find relationships between two variables, such as test scores or sales numbers.

💼 Career

Data scientists and analysts use Pearson correlation to explore data and find meaningful patterns that help in decision making.

1

Create exam score lists

Create two lists called exam1_scores and exam2_scores with these exact values: exam1_scores = [85, 90, 78, 92, 88] and exam2_scores = [80, 95, 75, 90, 85].

SciPy

# Create the two lists with the exact scores
# Your code here

Hint

Use square brackets to create lists and separate numbers with commas.

2

Import pearsonr function

Import the pearsonr function from the scipy.stats module.

SciPy

exam1_scores = [85, 90, 78, 92, 88]
exam2_scores = [80, 95, 75, 90, 85]

# Import pearsonr from scipy.stats
# Your code here

Hint

Use the syntax from scipy.stats import pearsonr to import the function.

3

Calculate Pearson correlation

Use the pearsonr function with exam1_scores and exam2_scores as arguments. Store the first returned value (correlation coefficient) in a variable called correlation.

SciPy

from scipy.stats import pearsonr

exam1_scores = [85, 90, 78, 92, 88]
exam2_scores = [80, 95, 75, 90, 85]

# Calculate Pearson correlation coefficient
# Your code here

Hint

The pearsonr function returns two values: correlation and p-value. Use correlation, _ = pearsonr(...) to get only the correlation.

4

Print the Pearson correlation coefficient

Print the value of the variable correlation.

SciPy

from scipy.stats import pearsonr

exam1_scores = [85, 90, 78, 92, 88]
exam2_scores = [80, 95, 75, 90, 85]

correlation, _ = pearsonr(exam1_scores, exam2_scores)

# Print the correlation coefficient
# Your code here

Hint

Use print(correlation) to display the result.

Practice

1. What does the Pearson correlation coefficient measure between two variables?

easy

A. The frequency of values in each variable

B. The difference in means between two groups

C. The total sum of values in each variable

D. The strength and direction of a linear relationship

Solution

1. Step 1: Understand Pearson correlation purpose

   Pearson correlation measures how two variables move together in a straight line.

2. Step 2: Identify what it quantifies

   It quantifies both strength (how close points are to a line) and direction (positive or negative slope).

3. Final Answer:

   The strength and direction of a linear relationship -> Option D

4. Quick Check:

   Pearson correlation = strength and direction [OK]

Hint: Pearson = linear relationship strength and direction [OK]

Common Mistakes:

• Confusing correlation with difference in means
• Thinking it measures frequency counts
• Assuming it measures total sums

2. Which of the following is the correct way to import the Pearson correlation function from scipy?

easy

A. from scipy.stats import pearsonr

B. import scipy.pearsonr

C. from scipy import pearsonr

D. import pearsonr from scipy.stats

Solution

1. Step 1: Recall scipy.stats module structure

   The pearsonr function is inside scipy.stats, so import must reflect that.

2. Step 2: Check correct import syntax

   Python import syntax for functions is 'from module import function', so 'from scipy.stats import pearsonr' is correct.

3. Final Answer:

   from scipy.stats import pearsonr -> Option A

4. Quick Check:

   Correct import syntax = from scipy.stats import pearsonr [OK]

Hint: Use 'from scipy.stats import pearsonr' to import [OK]

Common Mistakes:

• Using 'import scipy.pearsonr' which is invalid
• Trying 'from scipy import pearsonr' missing stats
• Wrong syntax like 'import pearsonr from scipy.stats'

3. What is the output of this code?

from scipy.stats import pearsonr
x = [1, 2, 3, 4, 5]
y = [2, 4, 6, 8, 10]
corr, p = pearsonr(x, y)
print("{:.2f}".format(corr))

medium

A. 0.00

B. -1.00

C. 1.00

D. 0.50

Solution

1. Step 1: Analyze the data relationship

   y is exactly 2 times x, so they have a perfect positive linear relationship.

2. Step 2: Understand Pearson correlation output

   Perfect positive linear relationship means correlation coefficient is 1.0.

3. Final Answer:

   1.00 -> Option C

4. Quick Check:

   Perfect positive linear correlation = 1.00 [OK]

Hint: Perfect linear scaling means correlation = 1.00 [OK]

Common Mistakes:

• Confusing correlation with p-value
• Expecting 0.5 for proportional data
• Ignoring perfect linear relationship

4. Identify the error in this code snippet:

from scipy.stats import pearsonr
x = [1, 2, 3]
y = [4, 5]
corr, p = pearsonr(x, y)
print(corr)

medium

A. Lists x and y have different lengths

B. pearsonr function is not imported correctly

C. Missing parentheses in print statement

D. Variables x and y must be numpy arrays

Solution

1. Step 1: Check input data lengths

   pearsonr requires x and y to have the same length. Here, x has 3 elements, y has 2.

2. Step 2: Understand error cause

   Different lengths cause a ValueError when calling pearsonr.

3. Final Answer:

   Lists x and y have different lengths -> Option A

4. Quick Check:

   Input length mismatch causes error [OK]

Hint: Ensure x and y lists have equal length [OK]

Common Mistakes:

• Assuming pearsonr accepts different length inputs
• Thinking print syntax is wrong in Python 3
• Believing inputs must be numpy arrays

5. You have two variables: temperature (in Celsius) and ice cream sales. You want to check if higher temperature relates to higher sales. Which code correctly computes the Pearson correlation and interprets a strong positive relationship?

hard

A. from scipy.stats import pearsonr
corr, p = pearsonr(temp, sales)
if p > 0.7:
print('Strong positive correlation')

B. from scipy.stats import pearsonr
corr, p = pearsonr(temp, sales)
if corr > 0.7:
print('Strong positive correlation')

C. from scipy.stats import pearsonr
corr, p = pearsonr(temp, sales)
if corr < -0.7:
print('Strong positive correlation')

D. from scipy.stats import pearsonr
corr, p = pearsonr(temp, sales)
if abs(corr) < 0.3:
print('Strong positive correlation')

Solution

1. Step 1: Calculate Pearson correlation

   Use pearsonr to get correlation coefficient and p-value between temperature and sales.

2. Step 2: Interpret correlation coefficient

   A strong positive correlation means corr > 0.7. Checking p-value > 0.7 or negative corr is incorrect.

3. Final Answer:

   from scipy.stats import pearsonr corr, p = pearsonr(temp, sales) if corr > 0.7: print('Strong positive correlation') -> Option B

4. Quick Check:

   Strong positive correlation if corr > 0.7 [OK]

Hint: Check corr > 0.7 for strong positive relation [OK]

Common Mistakes:

• Using p-value instead of correlation coefficient
• Checking for negative correlation for positive relation
• Using absolute value less than threshold incorrectly