Correlation analysis in ML Python - Model Metrics & Evaluation

Metrics & Evaluation - Correlation analysis

Which metric matters for Correlation analysis and WHY

Correlation analysis measures how two variables move together. The key metric is the correlation coefficient, which ranges from -1 to 1.

A value near 1 means a strong positive link (both go up together). Near -1 means a strong negative link (one goes up, the other down). Near 0 means no clear link.

This metric helps us understand relationships before building models. It shows if variables are related and how strongly.

Confusion matrix or equivalent visualization

Correlation analysis does not use a confusion matrix because it is not a classification task.

Instead, we use a correlation matrix to show pairwise correlations between variables.

      | Var1 | Var2 | Var3 |
    -------------------------
    Var1 |  1.0 | 0.85 | -0.3 |
    Var2 | 0.85 |  1.0 | -0.1 |
    Var3 | -0.3 | -0.1 |  1.0 |
    

Each number shows how strongly two variables move together.

Tradeoff: Interpreting correlation strength and causation

High correlation means variables move together, but it does NOT mean one causes the other.

For example, ice cream sales and sunglasses sales may both rise in summer (high correlation) but one does not cause the other.

We must be careful not to assume causation from correlation alone.

What "good" vs "bad" correlation values look like

A "good" correlation depends on the goal:

• Strong positive or negative correlation (above 0.7 or below -0.7) means variables are closely linked.
• Weak correlation (near 0) means little or no linear relationship.

For feature selection, strong correlations can help pick useful variables.

However, very high correlation between features (above 0.9) can cause redundancy and hurt some models.

Common pitfalls in correlation analysis

• Confusing correlation with causation: Just because two things move together does not mean one causes the other.
• Ignoring non-linear relationships: Correlation measures linear links only. Variables can be related in other ways.
• Outliers affecting correlation: Extreme values can distort correlation values.
• Data leakage: Using future or target data in correlation can give misleadingly high values.

Self-check question

Your correlation analysis shows a coefficient of 0.95 between two features. Is this always good?

Answer: Not always. Such a high correlation means the features are very similar. Using both may cause redundancy and confuse some models. You might want to keep only one to avoid overlap.

Key Result

Correlation coefficient shows strength and direction of linear relationship between variables, ranging from -1 to 1.

Practice

1. What does a correlation value of 0 indicate between two variables?

easy

A. There is no linear relationship between the variables

B. The variables move exactly together in the same direction

C. One variable increases while the other decreases perfectly

D. The variables have a perfect nonlinear relationship

Solution

1. Step 1: Understand correlation meaning

   Correlation measures linear relationship strength between two variables.

2. Step 2: Interpret zero correlation

   A correlation of 0 means no linear relationship exists between the variables.

3. Final Answer:

   There is no linear relationship between the variables -> Option A

4. Quick Check:

   Correlation = 0 means no linear link [OK]

Hint: Zero correlation means no linear link between variables [OK]

Common Mistakes:

• Thinking zero means variables are unrelated in any way
• Confusing zero correlation with negative correlation
• Assuming zero means perfect nonlinear relationship

2. Which of the following is the correct Python syntax to calculate Pearson correlation between two lists x and y using NumPy?

easy

A. np.correlation(x, y)

B. np.corrcoef(x, y)[0, 1]

C. np.pearson(x, y)

D. np.corr(x, y)

Solution

1. Step 1: Recall NumPy correlation function

   NumPy uses np.corrcoef() to compute correlation matrix.

2. Step 2: Extract Pearson correlation value

   The correlation between x and y is at position [0, 1] in the matrix.

3. Final Answer:

   np.corrcoef(x, y)[0, 1] -> Option B

4. Quick Check:

   Use np.corrcoef and index [0,1] for Pearson [OK]

Hint: Use np.corrcoef(x, y)[0,1] to get Pearson correlation [OK]

Common Mistakes:

• Using non-existent functions like np.correlation or np.pearson
• Forgetting to index the correlation matrix
• Confusing correlation with covariance functions

3. Given the code below, what is the output?

import numpy as np
x = [1, 2, 3, 4, 5]
y = [2, 4, 6, 8, 10]
correlation = np.corrcoef(x, y)[0, 1]
print("{:.2f}".format(round(correlation, 2)))

medium

A. 1.00

B. -1.00

C. 0.00

D. 0.50

Solution

1. Step 1: Analyze the data relationship

   List y is exactly twice x, so they have a perfect positive linear relationship.

2. Step 2: Calculate correlation

   Perfect positive linear relationship means correlation = 1.0.

3. Final Answer:

   1.00 -> Option A

4. Quick Check:

   Perfect positive linear relation = 1.00 [OK]

Hint: Perfect multiples have correlation 1.00 [OK]

Common Mistakes:

• Thinking correlation is 0 because values differ
• Confusing correlation with covariance magnitude
• Rounding errors causing wrong output

4. Identify the error in the following code snippet for calculating correlation:

import numpy as np
x = [1, 2, 3]
y = [4, 5]
correlation = np.corrcoef(x, y)[0, 1]
print(correlation)

medium

A. Missing import statement for correlation function

B. np.corrcoef requires pandas Series, not lists

C. Lists x and y have different lengths causing an error

D. Indexing [0, 1] is incorrect for correlation matrix

Solution

1. Step 1: Check input data lengths

   Lists x and y have lengths 3 and 2, which must be equal for correlation.

2. Step 2: Understand np.corrcoef requirements

   np.corrcoef requires inputs of equal length arrays to compute correlation matrix.

3. Final Answer:

   Lists x and y have different lengths causing an error -> Option C

4. Quick Check:

   Input arrays must be same length [OK]

Hint: Check input lengths match before correlation [OK]

Common Mistakes:

• Assuming np.corrcoef works with unequal length lists
• Thinking pandas Series are required for np.corrcoef
• Misindexing the correlation matrix

5. You have two features: hours_studied and exam_score. You calculate their correlation and get -0.85. What does this imply and what should you consider next?

hard

A. Perfect negative nonlinear relationship; use polynomial regression

B. Strong positive linear relationship; increase study hours to improve score

C. No relationship; ignore this feature in modeling

D. Strong negative linear relationship; consider if data or labels are reversed

Solution

1. Step 1: Interpret correlation sign and magnitude

   A correlation of -0.85 means a strong negative linear relationship between the two features.

2. Step 2: Consider real-world meaning

   Since more study usually means higher scores, a strong negative correlation suggests possible data issues or reversed labels.

3. Final Answer:

   Strong negative linear relationship; consider if data or labels are reversed -> Option D

4. Quick Check:

   Negative strong correlation may indicate reversed data [OK]

Hint: Strong negative correlation may mean reversed data [OK]

Common Mistakes:

• Assuming negative correlation means no relationship
• Ignoring domain knowledge about expected relationships
• Confusing negative linear with nonlinear relationships