Pearson correlation in SciPy - Cheat Sheet & Quick Revision

Recall & Review

beginner

What does the Pearson correlation coefficient measure?

It measures the strength and direction of a linear relationship between two variables.

beginner

What is the range of values for the Pearson correlation coefficient?

The values range from -1 to 1, where 1 means perfect positive linear correlation, -1 means perfect negative linear correlation, and 0 means no linear correlation.

beginner

How do you calculate Pearson correlation using scipy?

Use scipy.stats.pearsonr(x, y) where x and y are arrays of data. It returns the correlation coefficient and the p-value.

intermediate

What does a p-value returned by scipy.stats.pearsonr indicate?

It indicates the probability that the observed correlation happened by chance. A small p-value (e.g., < 0.05) suggests the correlation is statistically significant.

intermediate

Can Pearson correlation detect non-linear relationships?

No, Pearson correlation only measures linear relationships. Non-linear relationships may have a low or zero Pearson correlation even if variables are related.

What does a Pearson correlation coefficient of 0.8 indicate?

A Strong positive linear relationship

B Strong negative linear relationship

C No linear relationship

D Perfect negative linear relationship

A value of 0.8 means a strong positive linear relationship between the two variables.

Which scipy function calculates Pearson correlation?

A scipy.stats.spearmanr

B scipy.stats.kendalltau

C scipy.stats.ttest_ind

D scipy.stats.pearsonr

scipy.stats.pearsonr calculates the Pearson correlation coefficient and p-value.

What does a p-value less than 0.05 mean in Pearson correlation test?

A No correlation exists

B Correlation is likely due to chance

C Correlation is statistically significant

D Data is not normally distributed

A p-value less than 0.05 suggests the correlation is statistically significant and unlikely due to chance.

What is the Pearson correlation coefficient for no linear relationship?

A 0

B -1

C 0.5

D 1

A coefficient of 0 means no linear relationship between variables.

Can Pearson correlation detect a curved (non-linear) relationship?

A Only if data is large

B No, it only measures linear relationships

C Yes, always

D Only if variables are categorical

Pearson correlation only measures linear relationships and may miss non-linear associations.

Explain how to use scipy to calculate Pearson correlation between two data arrays.

Think about the function name and what it returns.

Describe what the Pearson correlation coefficient tells us about two variables.

Focus on what the number means in terms of relationship.

Practice

1. What does the Pearson correlation coefficient measure between two variables?

easy

A. The frequency of values in each variable

B. The difference in means between two groups

C. The total sum of values in each variable

D. The strength and direction of a linear relationship

Solution

1. Step 1: Understand Pearson correlation purpose

   Pearson correlation measures how two variables move together in a straight line.

2. Step 2: Identify what it quantifies

   It quantifies both strength (how close points are to a line) and direction (positive or negative slope).

3. Final Answer:

   The strength and direction of a linear relationship -> Option D

4. Quick Check:

   Pearson correlation = strength and direction [OK]

Hint: Pearson = linear relationship strength and direction [OK]

Common Mistakes:

• Confusing correlation with difference in means
• Thinking it measures frequency counts
• Assuming it measures total sums

2. Which of the following is the correct way to import the Pearson correlation function from scipy?

easy

A. from scipy.stats import pearsonr

B. import scipy.pearsonr

C. from scipy import pearsonr

D. import pearsonr from scipy.stats

Solution

1. Step 1: Recall scipy.stats module structure

   The pearsonr function is inside scipy.stats, so import must reflect that.

2. Step 2: Check correct import syntax

   Python import syntax for functions is 'from module import function', so 'from scipy.stats import pearsonr' is correct.

3. Final Answer:

   from scipy.stats import pearsonr -> Option A

4. Quick Check:

   Correct import syntax = from scipy.stats import pearsonr [OK]

Hint: Use 'from scipy.stats import pearsonr' to import [OK]

Common Mistakes:

• Using 'import scipy.pearsonr' which is invalid
• Trying 'from scipy import pearsonr' missing stats
• Wrong syntax like 'import pearsonr from scipy.stats'

3. What is the output of this code?

from scipy.stats import pearsonr
x = [1, 2, 3, 4, 5]
y = [2, 4, 6, 8, 10]
corr, p = pearsonr(x, y)
print("{:.2f}".format(corr))

medium

A. 0.00

B. -1.00

C. 1.00

D. 0.50

Solution

1. Step 1: Analyze the data relationship

   y is exactly 2 times x, so they have a perfect positive linear relationship.

2. Step 2: Understand Pearson correlation output

   Perfect positive linear relationship means correlation coefficient is 1.0.

3. Final Answer:

   1.00 -> Option C

4. Quick Check:

   Perfect positive linear correlation = 1.00 [OK]

Hint: Perfect linear scaling means correlation = 1.00 [OK]

Common Mistakes:

• Confusing correlation with p-value
• Expecting 0.5 for proportional data
• Ignoring perfect linear relationship

4. Identify the error in this code snippet:

from scipy.stats import pearsonr
x = [1, 2, 3]
y = [4, 5]
corr, p = pearsonr(x, y)
print(corr)

medium

A. Lists x and y have different lengths

B. pearsonr function is not imported correctly

C. Missing parentheses in print statement

D. Variables x and y must be numpy arrays

Solution

1. Step 1: Check input data lengths

   pearsonr requires x and y to have the same length. Here, x has 3 elements, y has 2.

2. Step 2: Understand error cause

   Different lengths cause a ValueError when calling pearsonr.

3. Final Answer:

   Lists x and y have different lengths -> Option A

4. Quick Check:

   Input length mismatch causes error [OK]

Hint: Ensure x and y lists have equal length [OK]

Common Mistakes:

• Assuming pearsonr accepts different length inputs
• Thinking print syntax is wrong in Python 3
• Believing inputs must be numpy arrays

5. You have two variables: temperature (in Celsius) and ice cream sales. You want to check if higher temperature relates to higher sales. Which code correctly computes the Pearson correlation and interprets a strong positive relationship?

hard

A. from scipy.stats import pearsonr
corr, p = pearsonr(temp, sales)
if p > 0.7:
print('Strong positive correlation')

B. from scipy.stats import pearsonr
corr, p = pearsonr(temp, sales)
if corr > 0.7:
print('Strong positive correlation')

C. from scipy.stats import pearsonr
corr, p = pearsonr(temp, sales)
if corr < -0.7:
print('Strong positive correlation')

D. from scipy.stats import pearsonr
corr, p = pearsonr(temp, sales)
if abs(corr) < 0.3:
print('Strong positive correlation')

Solution

1. Step 1: Calculate Pearson correlation

   Use pearsonr to get correlation coefficient and p-value between temperature and sales.

2. Step 2: Interpret correlation coefficient

   A strong positive correlation means corr > 0.7. Checking p-value > 0.7 or negative corr is incorrect.

3. Final Answer:

   from scipy.stats import pearsonr corr, p = pearsonr(temp, sales) if corr > 0.7: print('Strong positive correlation') -> Option B

4. Quick Check:

   Strong positive correlation if corr > 0.7 [OK]

Hint: Check corr > 0.7 for strong positive relation [OK]

Common Mistakes:

• Using p-value instead of correlation coefficient
• Checking for negative correlation for positive relation
• Using absolute value less than threshold incorrectly