Pearson correlation in SciPy - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Calculate Pearson correlation coefficient

What is the output of this code that calculates the Pearson correlation coefficient between two lists?

SciPy

from scipy.stats import pearsonr
x = [1, 2, 3, 4, 5]
y = [2, 4, 6, 8, 10]
result = pearsonr(x, y)
print(round(result[0], 2))

A 1.00

B 0.00

C -1.00

D 0.50

💡 Hint

Pearson correlation measures linear relationship strength between two variables.

The lists x and y have a perfect positive linear relationship (y is exactly 2 times x). So, the Pearson correlation coefficient is 1.00.

❓ data_output

intermediate

Number of pairs with strong positive correlation

Given this DataFrame, how many pairs of columns have a Pearson correlation coefficient greater than 0.8?

SciPy

import pandas as pd
import numpy as np
np.random.seed(0)
data = pd.DataFrame({
  'A': np.random.rand(10),
  'B': np.random.rand(10),
  'C': np.linspace(0, 1, 10),
  'D': np.linspace(0, 1, 10) * 2
})
corr_matrix = data.corr()
strong_pairs = (corr_matrix.where(np.triu(np.ones(corr_matrix.shape), k=1).astype(bool)) > 0.8).sum().sum()
print(int(strong_pairs))

A 2

B 0

C 3

D 1

💡 Hint

Check pairs only once and ignore diagonal.

Columns C and D are perfectly correlated (correlation 1). A and B have low correlation (~ -0.03), and other cross pairs are also low. The upper triangular mask counts unique off-diagonal pairs once, so only 1 pair (C, D).

🔧 Debug

advanced

Identify the error in Pearson correlation calculation

What error does this code raise when trying to calculate Pearson correlation?

SciPy

from scipy.stats import pearsonr
x = [1, 2, 3, 4, 5]
y = [2, 4, 6, 8]
pearsonr(x, y)

A IndexError: list index out of range

B TypeError: unsupported operand type(s) for -: 'list' and 'list'

C ValueError: x and y must have the same length

D No error, returns correlation

💡 Hint

Check if both lists have the same number of elements.

The lists x and y have different lengths (5 and 4). pearsonr requires equal length inputs and raises ValueError if lengths differ.

🚀 Application

advanced

Interpret Pearson correlation result

If the Pearson correlation coefficient between two variables is -0.85, what does this mean?

A Strong negative linear relationship

B Perfect positive linear relationship

C No linear relationship

D Strong positive linear relationship

💡 Hint

Negative values indicate inverse relationships.

A correlation of -0.85 means a strong negative linear relationship: as one variable increases, the other tends to decrease.

🧠 Conceptual

expert

Effect of outliers on Pearson correlation

Which statement best describes how outliers affect the Pearson correlation coefficient?

A Outliers always increase the Pearson correlation value

B Outliers can greatly distort the Pearson correlation, making it unreliable

C Outliers have little to no effect on Pearson correlation

D Outliers convert Pearson correlation into a non-linear measure

💡 Hint

Think about how extreme values influence averages and covariance.

Pearson correlation is sensitive to outliers because it uses means and standard deviations. Extreme values can skew the correlation, making it misleading.

Practice

1. What does the Pearson correlation coefficient measure between two variables?

easy

A. The frequency of values in each variable

B. The difference in means between two groups

C. The total sum of values in each variable

D. The strength and direction of a linear relationship

Solution

1. Step 1: Understand Pearson correlation purpose

   Pearson correlation measures how two variables move together in a straight line.

2. Step 2: Identify what it quantifies

   It quantifies both strength (how close points are to a line) and direction (positive or negative slope).

3. Final Answer:

   The strength and direction of a linear relationship -> Option D

4. Quick Check:

   Pearson correlation = strength and direction [OK]

Hint: Pearson = linear relationship strength and direction [OK]

Common Mistakes:

• Confusing correlation with difference in means
• Thinking it measures frequency counts
• Assuming it measures total sums

2. Which of the following is the correct way to import the Pearson correlation function from scipy?

easy

A. from scipy.stats import pearsonr

B. import scipy.pearsonr

C. from scipy import pearsonr

D. import pearsonr from scipy.stats

Solution

1. Step 1: Recall scipy.stats module structure

   The pearsonr function is inside scipy.stats, so import must reflect that.

2. Step 2: Check correct import syntax

   Python import syntax for functions is 'from module import function', so 'from scipy.stats import pearsonr' is correct.

3. Final Answer:

   from scipy.stats import pearsonr -> Option A

4. Quick Check:

   Correct import syntax = from scipy.stats import pearsonr [OK]

Hint: Use 'from scipy.stats import pearsonr' to import [OK]

Common Mistakes:

• Using 'import scipy.pearsonr' which is invalid
• Trying 'from scipy import pearsonr' missing stats
• Wrong syntax like 'import pearsonr from scipy.stats'

3. What is the output of this code?

from scipy.stats import pearsonr
x = [1, 2, 3, 4, 5]
y = [2, 4, 6, 8, 10]
corr, p = pearsonr(x, y)
print("{:.2f}".format(corr))

medium

A. 0.00

B. -1.00

C. 1.00

D. 0.50

Solution

1. Step 1: Analyze the data relationship

   y is exactly 2 times x, so they have a perfect positive linear relationship.

2. Step 2: Understand Pearson correlation output

   Perfect positive linear relationship means correlation coefficient is 1.0.

3. Final Answer:

   1.00 -> Option C

4. Quick Check:

   Perfect positive linear correlation = 1.00 [OK]

Hint: Perfect linear scaling means correlation = 1.00 [OK]

Common Mistakes:

• Confusing correlation with p-value
• Expecting 0.5 for proportional data
• Ignoring perfect linear relationship

4. Identify the error in this code snippet:

from scipy.stats import pearsonr
x = [1, 2, 3]
y = [4, 5]
corr, p = pearsonr(x, y)
print(corr)

medium

A. Lists x and y have different lengths

B. pearsonr function is not imported correctly

C. Missing parentheses in print statement

D. Variables x and y must be numpy arrays

Solution

1. Step 1: Check input data lengths

   pearsonr requires x and y to have the same length. Here, x has 3 elements, y has 2.

2. Step 2: Understand error cause

   Different lengths cause a ValueError when calling pearsonr.

3. Final Answer:

   Lists x and y have different lengths -> Option A

4. Quick Check:

   Input length mismatch causes error [OK]

Hint: Ensure x and y lists have equal length [OK]

Common Mistakes:

• Assuming pearsonr accepts different length inputs
• Thinking print syntax is wrong in Python 3
• Believing inputs must be numpy arrays

5. You have two variables: temperature (in Celsius) and ice cream sales. You want to check if higher temperature relates to higher sales. Which code correctly computes the Pearson correlation and interprets a strong positive relationship?

hard

A. from scipy.stats import pearsonr
corr, p = pearsonr(temp, sales)
if p > 0.7:
print('Strong positive correlation')

B. from scipy.stats import pearsonr
corr, p = pearsonr(temp, sales)
if corr > 0.7:
print('Strong positive correlation')

C. from scipy.stats import pearsonr
corr, p = pearsonr(temp, sales)
if corr < -0.7:
print('Strong positive correlation')

D. from scipy.stats import pearsonr
corr, p = pearsonr(temp, sales)
if abs(corr) < 0.3:
print('Strong positive correlation')

Solution

1. Step 1: Calculate Pearson correlation

   Use pearsonr to get correlation coefficient and p-value between temperature and sales.

2. Step 2: Interpret correlation coefficient

   A strong positive correlation means corr > 0.7. Checking p-value > 0.7 or negative corr is incorrect.

3. Final Answer:

   from scipy.stats import pearsonr corr, p = pearsonr(temp, sales) if corr > 0.7: print('Strong positive correlation') -> Option B

4. Quick Check:

   Strong positive correlation if corr > 0.7 [OK]

Hint: Check corr > 0.7 for strong positive relation [OK]

Common Mistakes:

• Using p-value instead of correlation coefficient
• Checking for negative correlation for positive relation
• Using absolute value less than threshold incorrectly