P-values and significance in Data Analysis Python - Mini Project: Build & Apply

Understanding P-values and Significance in Data Analysis

📖 Scenario: You are a data analyst working with a small dataset of exam scores from two different teaching methods. You want to find out if the difference in average scores between the two methods is significant or could have happened by chance.

🎯 Goal: Build a simple Python program that calculates the p-value from two groups of exam scores and decides if the difference is statistically significant.

📋 What You'll Learn

Create two lists of exam scores for Method A and Method B with exact values

Set a significance level variable called alpha to 0.05

Use the scipy.stats library to perform an independent t-test

Print the p-value and a message stating if the difference is significant or not

💡 Why This Matters

🌍 Real World

Scientists and analysts use p-values to decide if their findings are likely real or just random chance.

💼 Career

Understanding p-values is essential for data analysts, researchers, and anyone interpreting statistical test results.

1

Create exam score lists for two teaching methods

Create two lists called method_a_scores and method_b_scores with these exact values: method_a_scores = [88, 92, 85, 91, 87] and method_b_scores = [78, 81, 79, 77, 80].

Data Analysis Python

# Create two lists for exam scores of Method A and Method B
# Your code here

Hint

Use square brackets to create lists and separate numbers with commas.

2

Set the significance level

Create a variable called alpha and set it to 0.05 to represent the significance level.

Data Analysis Python

method_a_scores = [88, 92, 85, 91, 87]
method_b_scores = [78, 81, 79, 77, 80]
# Set the significance level alpha to 0.05
# Your code here

Hint

The significance level is usually set to 0.05 in many tests.

3

Perform an independent t-test to calculate the p-value

Import ttest_ind from scipy.stats and use it to calculate the t-test between method_a_scores and method_b_scores. Store the p-value in a variable called p_value.

Data Analysis Python

method_a_scores = [88, 92, 85, 91, 87]
method_b_scores = [78, 81, 79, 77, 80]
alpha = 0.05
# Import ttest_ind and calculate p_value
# Your code here

Hint

Use from scipy.stats import ttest_ind and then call ttest_ind(list1, list2).

4

Print the p-value and significance result

Write code to print the p-value with the text "P-value:". Then use an if statement to print "The difference is significant." if p_value is less than alpha, otherwise print "The difference is not significant.".

Data Analysis Python

from scipy.stats import ttest_ind

method_a_scores = [88, 92, 85, 91, 87]
method_b_scores = [78, 81, 79, 77, 80]
alpha = 0.05

stat, p_value = ttest_ind(method_a_scores, method_b_scores)
# Print the p-value and check if it is less than alpha
# Your code here

Hint

Use print(f"P-value: {p_value}") and an if statement to compare p_value and alpha.

Practice

1. What does a p-value less than 0.05 usually indicate in a hypothesis test?

easy

A. The results are statistically significant and unlikely due to chance.

B. The results are definitely caused by random chance.

C. The sample size is too small to draw conclusions.

D. The test was performed incorrectly.

Solution

1. Step 1: Understand the meaning of p-value

   A p-value measures how likely the observed data would occur if the null hypothesis were true (random chance).

2. Step 2: Compare p-value to significance level

   If the p-value is less than 0.05 (common threshold), it means the observed result is unlikely due to chance, so we reject the null hypothesis.

3. Final Answer:

   The results are statistically significant and unlikely due to chance. -> Option A

4. Quick Check:

   p-value < 0.05 means significant [OK]

Hint: p-value < 0.05 means strong evidence against chance [OK]

Common Mistakes:

• Thinking p-value shows probability null is true
• Confusing significance level with p-value
• Assuming p-value proves practical importance

2. Which of the following is the correct way to check if a p-value stored in variable p_val is significant at 0.05 level in Python?

easy

A. if p_val == 0.05: print('Significant')

B. if p_val > 0.05: print('Significant')

C. if p_val < 0.05: print('Significant')

D. if p_val <= 0.5: print('Significant')

Solution

1. Step 1: Recall significance condition

   A p-value less than 0.05 means significance, so we check if p_val < 0.05.

2. Step 2: Match condition to code

   if p_val < 0.05: print('Significant') uses < 0.05 correctly. Others check wrong conditions or wrong thresholds.

3. Final Answer:

   if p_val < 0.05: print('Significant') -> Option C

4. Quick Check:

   Use < 0.05 for significance [OK]

Hint: Check if p_val < 0.05 for significance [OK]

Common Mistakes:

• Using > instead of < for significance
• Checking equality to 0.05 exactly
• Using wrong threshold like 0.5

3. Given this Python code snippet, what will be printed?

p_val = 0.07
if p_val < 0.05:
    print('Reject null hypothesis')
else:
    print('Fail to reject null hypothesis')

medium

A. Reject null hypothesis

B. SyntaxError

C. No output

D. Fail to reject null hypothesis

Solution

1. Step 1: Check the p-value against 0.05

   The p_val is 0.07, which is greater than 0.05, so the condition p_val < 0.05 is False.

2. Step 2: Determine which branch runs

   Since the condition is False, the else block runs, printing 'Fail to reject null hypothesis'.

3. Final Answer:

   Fail to reject null hypothesis -> Option D

4. Quick Check:

   p_val = 0.07 > 0.05 means fail to reject [OK]

Hint: p_val > 0.05 means fail to reject null [OK]

Common Mistakes:

• Confusing greater than with less than
• Assuming equality means reject
• Ignoring else branch

4. This code is intended to print 'Significant' if p-value is less than 0.05, but it always prints 'Not Significant'. What is the error?

p_val = 0.03
if p_val > 0.05:
    print('Significant')
else:
    print('Not Significant')

medium

A. The comparison operator should be '<' instead of '>'

B. The variable p_val is not defined

C. The print statements are reversed

D. The p_val value is incorrect

Solution

1. Step 1: Analyze the condition logic

   The code checks if p_val > 0.05 to print 'Significant', but significance means p_val < 0.05.

2. Step 2: Identify the fix

   Changing the operator to < will correctly print 'Significant' when p_val is 0.03.

3. Final Answer:

   The comparison operator should be '<' instead of '>' -> Option A

4. Quick Check:

   Use < 0.05 for significance check [OK]

Hint: Use '<' to check p-value significance [OK]

Common Mistakes:

• Using '>' instead of '<' for significance
• Swapping print statements without fixing condition
• Assuming variable is undefined

5. You run a test and get a p-value of 0.049. You want to be very strict and use a significance level of 0.01. What should you conclude?

hard

A. Reject the null hypothesis because 0.049 < 0.05

B. Fail to reject the null hypothesis because 0.049 > 0.01

C. Reject the null hypothesis because 0.049 < 0.01

D. Fail to reject the null hypothesis because 0.049 > 0.05

Solution

1. Step 1: Understand significance level

   The significance level is set to 0.01, meaning p-values must be less than 0.01 to reject the null hypothesis.

2. Step 2: Compare p-value to significance level

   The p-value 0.049 is greater than 0.01, so we fail to reject the null hypothesis at this stricter level.

3. Final Answer:

   Fail to reject the null hypothesis because 0.049 > 0.01 -> Option B

4. Quick Check:

   p-value > significance level means fail to reject [OK]

Hint: Compare p-value to chosen significance level carefully [OK]

Common Mistakes:

• Using default 0.05 without checking chosen level
• Confusing less than and greater than
• Ignoring stricter significance level