Spearman correlation in SciPy - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Spearman correlation output for simple arrays

What is the output of the following code that calculates Spearman correlation between two arrays?

SciPy

from scipy.stats import spearmanr
x = [1, 2, 3, 4, 5]
y = [5, 6, 7, 8, 7]
correlation, pvalue = spearmanr(x, y)
print(round(correlation, 2))

A 1.0

B -0.9

C 0.7

D 0.9

💡 Hint

Spearman correlation measures monotonic relationship, not linear.

The Spearman correlation here is about 0.9, showing a strong positive monotonic relationship.

❓ data_output

intermediate

Spearman correlation matrix from DataFrame columns

Given a DataFrame with columns A, B, and C, what is the Spearman correlation matrix output?

SciPy

import pandas as pd
from scipy.stats import spearmanr

data = {'A': [1, 2, 3, 4], 'B': [4, 3, 2, 1], 'C': [1, 3, 2, 4]}
df = pd.DataFrame(data)
correlation, _ = spearmanr(df)
print(correlation)

A

[[ 1.  -1.   0.8]
 [-1.   1.  -0.8]
 [ 0.8 -0.8  1. ]]

B

[[ 1.   1.  -0.8]
 [ 1.   1.  -0.8]
 [-0.8 -0.8  1. ]]

C

[[ 1.   0.8 -1. ]
 [ 0.8  1.  -1. ]
 [-1.  -1.   1. ]]

D

[[ 1.   0.5  0.5]
 [ 0.5  1.   0.5]
 [ 0.5  0.5  1. ]]

💡 Hint

Spearman correlation matrix is symmetric with 1s on diagonal.

Column A and B are perfectly negatively correlated (-1), A and C have 0.8 positive correlation.

🔧 Debug

advanced

Identify the error in Spearman correlation code

What error does this code raise when trying to calculate Spearman correlation?

SciPy

from scipy.stats import spearmanr
x = [1, 2, 3]
y = [4, 5]
spearmanr(x, y)

A ValueError: x and y must have the same length

B TypeError: spearmanr() missing 1 required positional argument

C IndexError: list index out of range

D No error, returns correlation

💡 Hint

Check if input arrays have the same length.

Spearman correlation requires inputs of equal length; otherwise, ValueError is raised.

🧠 Conceptual

advanced

Understanding Spearman correlation properties

Which statement about Spearman correlation is TRUE?

A It is sensitive to outliers more than Pearson correlation.

B It measures linear relationships only.

C It uses ranks of data values to measure monotonic relationships.

D It requires data to be normally distributed.

💡 Hint

Think about how Spearman correlation handles data values.

Spearman correlation uses ranks to measure monotonic relationships, not just linear.

🚀 Application

expert

Interpreting Spearman correlation p-value

You calculate Spearman correlation between two variables and get correlation=0.6 and p-value=0.07. What is the correct interpretation?

A There is no relationship because p-value is greater than 0.05.

B There is a moderate monotonic relationship but it is NOT statistically significant at 0.05 level.

C There is a strong monotonic relationship and it is statistically significant at 0.05 level.

D The correlation is weak and p-value indicates strong significance.

💡 Hint

Check both correlation strength and p-value significance.

Correlation 0.6 is moderate positive, but p-value 0.07 is above 0.05 threshold, so not significant.

Practice

1. What does the Spearman correlation measure between two variables?

easy

A. The variance of a single variable

B. The strength and direction of a monotonic relationship based on ranks

C. The difference in means of two variables

D. The linear relationship between two variables

Solution

1. Step 1: Understand correlation types

   Spearman correlation measures how two variables move together based on their rank order, not their actual values.

2. Step 2: Compare with other correlations

   Unlike Pearson correlation, which measures linear relationships, Spearman focuses on monotonic relationships using ranks.

3. Final Answer:

   The strength and direction of a monotonic relationship based on ranks -> Option B

4. Quick Check:

   Spearman = monotonic rank correlation [OK]

Hint: Spearman uses ranks, not raw values, to find monotonic links [OK]

Common Mistakes:

• Confusing Spearman with Pearson correlation
• Thinking it measures linear relationships only
• Assuming it compares means or variances

2. Which of the following is the correct way to import the Spearman correlation function from scipy?

easy

A. from scipy.stats import spearmanr

B. import scipy.spearmanr

C. from scipy import spearmanr

D. import spearmanr from scipy.stats

Solution

1. Step 1: Recall scipy.stats module structure

   The Spearman correlation function is inside the stats submodule of scipy, so it must be imported from scipy.stats.

2. Step 2: Check correct import syntax

   The correct Python syntax to import a function is 'from module import function', so 'from scipy.stats import spearmanr' is correct.

3. Final Answer:

   from scipy.stats import spearmanr -> Option A

4. Quick Check:

   Correct import syntax = from scipy.stats import spearmanr [OK]

Hint: Use 'from scipy.stats import spearmanr' to import correctly [OK]

Common Mistakes:

• Trying to import spearmanr directly from scipy
• Using incorrect import syntax like 'import spearmanr from ...'
• Missing the stats submodule in import

3. What is the output of this code snippet?

from scipy.stats import spearmanr
x = [1, 2, 3, 4, 5]
y = [5, 6, 7, 8, 7]
result = spearmanr(x, y)
print(round(result.correlation, 2))

medium

A. 0.5

B. 0.9

C. 1.0

D. 0.82

Solution

1. Step 1: Understand the data ranks

   x is increasing from 1 to 5, y mostly increases but has a slight drop at the end (7 instead of 9), so ranks are similar but not perfect.

2. Step 2: Calculate Spearman correlation

   Spearman correlation measures rank similarity; here it is less than 1 but still strong, about 0.82 after rounding.

3. Final Answer:

   0.82 -> Option D

4. Quick Check:

   Spearman correlation ≈ 0.82 [OK]

Hint: Spearman close to 1 means strong rank similarity [OK]

Common Mistakes:

• Assuming perfect correlation (1.0) without checking data
• Confusing Pearson and Spearman correlation values
• Ignoring the slight rank difference in y

4. Identify the error in this code that calculates Spearman correlation:

from scipy.stats import spearmanr
x = [1, 2, 3]
y = [4, 5]
result = spearmanr(x, y)
print(result.correlation)

medium

A. Missing import for numpy

B. spearmanr function is misspelled

C. Lists x and y have different lengths

D. result.statistic attribute does not exist

Solution

1. Step 1: Check input data lengths

   Spearman correlation requires both input lists to have the same length. Here, x has 3 elements, y has 2.

2. Step 2: Understand function requirements

   Because lengths differ, spearmanr will raise an error. Other options are incorrect as spelling and attribute usage are correct.

3. Final Answer:

   Lists x and y have different lengths -> Option C

4. Quick Check:

   Input length mismatch causes error [OK]

Hint: Ensure input lists have equal length before correlation [OK]

Common Mistakes:

• Ignoring length mismatch of input lists
• Thinking spearmanr needs numpy import explicitly
• Assuming result.statistic is invalid

5. You have two variables with many tied ranks and some missing values (NaNs). Which approach using scipy.stats.spearmanr is best to get a valid Spearman correlation?

hard

A. Use spearmanr with nan_policy='omit' to ignore NaNs and handle ties automatically

B. Replace NaNs with zeros and then use spearmanr without extra parameters

C. Remove all tied values manually before using spearmanr

D. Use spearmanr with nan_policy='raise' to get an error for NaNs

Solution

1. Step 1: Handle missing values properly

   Spearman correlation in scipy can handle NaNs if nan_policy='omit' is set, which ignores NaNs safely.

2. Step 2: Understand tied ranks handling

   Spearman correlation automatically manages tied ranks, so no manual removal is needed.

3. Step 3: Evaluate other options

   Replacing NaNs with zeros or removing tied values manually can bias results; raising errors is not helpful for analysis.

4. Final Answer:

   Use spearmanr with nan_policy='omit' to ignore NaNs and handle ties automatically -> Option A

5. Quick Check:

   nan_policy='omit' + automatic tie handling = Use spearmanr with nan_policy='omit' to ignore NaNs and handle ties automatically [OK]

Hint: Use nan_policy='omit' to skip NaNs safely in spearmanr [OK]

Common Mistakes:

• Replacing NaNs with zeros causing bias
• Trying to remove tied ranks manually
• Not setting nan_policy causing errors