P-values and significance in Data Analysis Python - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Understanding P-value from a t-test

What is the output of the following Python code that performs a t-test on two small samples?

Data Analysis Python

from scipy.stats import ttest_ind
sample1 = [5, 7, 8, 6, 9]
sample2 = [10, 12, 11, 13, 14]
t_stat, p_value = ttest_ind(sample1, sample2)
print(round(p_value, 3))

A 0.05

B 0.002

C 0.5

D 0.1

💡 Hint

Think about how different the two samples are and what a small p-value means.

The t-test compares the means of two samples. Because the samples differ clearly, the p-value is very small, indicating strong evidence against the null hypothesis.

❓ data_output

intermediate

Interpreting significance from p-values in a DataFrame

Given the DataFrame below with p-values from multiple tests, which rows are considered statistically significant at alpha = 0.05?

Data Analysis Python

import pandas as pd
data = {'Test': ['A', 'B', 'C', 'D'], 'p_value': [0.03, 0.07, 0.001, 0.2]}
df = pd.DataFrame(data)
significant = df[df['p_value'] < 0.05]
print(significant['Test'].tolist())

A ['A', 'B', 'C']

B ['B', 'D']

C ['A', 'C']

D ['C', 'D']

💡 Hint

Remember significance means p-value less than 0.05.

Only tests A and C have p-values below 0.05, so they are significant.

🧠 Conceptual

advanced

Effect of sample size on p-value

Which statement best explains how increasing sample size affects the p-value in hypothesis testing?

A Increasing sample size decreases variability, which can lead to smaller p-values if the effect exists.

B Increasing sample size always increases the p-value, making results less significant.

C Sample size does not affect the p-value; it only depends on the observed effect size.

D Increasing sample size always leads to p-values equal to zero.

💡 Hint

Think about how more data affects the certainty of results.

Larger samples reduce random noise, making it easier to detect true effects, often lowering p-values if an effect is present.

🔧 Debug

advanced

Identifying error in p-value calculation code

What error will this code produce when trying to calculate a p-value from a t-test?

Data Analysis Python

from scipy.stats import ttest_ind
sample1 = [1, 2, 3]
sample2 = [4, 5]
t_stat, p_value = ttest_ind(sample1, sample2, equal_var=False)
print(p_value)

A IndexError due to list indexing

B ValueError due to unequal sample sizes

C TypeError because equal_var is not a valid argument

D No error; prints a valid p-value

💡 Hint

Check the documentation for ttest_ind about sample sizes and parameters.

The function supports unequal sample sizes and the equal_var parameter is valid, so no error occurs.

🚀 Application

expert

Choosing significance level for multiple tests

You run 20 independent hypothesis tests each at alpha = 0.05. What is the approximate probability of getting at least one false positive (Type I error) by chance?

A About 0.64

B About 0.95

C About 1.0

D Exactly 0.05

💡 Hint

Use the formula for the complement of no false positives: 1 - (1 - alpha)^number_of_tests.

The chance of no false positives is (1 - 0.05)^20 ≈ 0.358, so the chance of at least one false positive is about 1 - 0.358 = 0.642.

Practice

1. What does a p-value less than 0.05 usually indicate in a hypothesis test?

easy

A. The results are statistically significant and unlikely due to chance.

B. The results are definitely caused by random chance.

C. The sample size is too small to draw conclusions.

D. The test was performed incorrectly.

Solution

1. Step 1: Understand the meaning of p-value

   A p-value measures how likely the observed data would occur if the null hypothesis were true (random chance).

2. Step 2: Compare p-value to significance level

   If the p-value is less than 0.05 (common threshold), it means the observed result is unlikely due to chance, so we reject the null hypothesis.

3. Final Answer:

   The results are statistically significant and unlikely due to chance. -> Option A

4. Quick Check:

   p-value < 0.05 means significant [OK]

Hint: p-value < 0.05 means strong evidence against chance [OK]

Common Mistakes:

• Thinking p-value shows probability null is true
• Confusing significance level with p-value
• Assuming p-value proves practical importance

2. Which of the following is the correct way to check if a p-value stored in variable p_val is significant at 0.05 level in Python?

easy

A. if p_val == 0.05: print('Significant')

B. if p_val > 0.05: print('Significant')

C. if p_val < 0.05: print('Significant')

D. if p_val <= 0.5: print('Significant')

Solution

1. Step 1: Recall significance condition

   A p-value less than 0.05 means significance, so we check if p_val < 0.05.

2. Step 2: Match condition to code

   if p_val < 0.05: print('Significant') uses < 0.05 correctly. Others check wrong conditions or wrong thresholds.

3. Final Answer:

   if p_val < 0.05: print('Significant') -> Option C

4. Quick Check:

   Use < 0.05 for significance [OK]

Hint: Check if p_val < 0.05 for significance [OK]

Common Mistakes:

• Using > instead of < for significance
• Checking equality to 0.05 exactly
• Using wrong threshold like 0.5

3. Given this Python code snippet, what will be printed?

p_val = 0.07
if p_val < 0.05:
    print('Reject null hypothesis')
else:
    print('Fail to reject null hypothesis')

medium

A. Reject null hypothesis

B. SyntaxError

C. No output

D. Fail to reject null hypothesis

Solution

1. Step 1: Check the p-value against 0.05

   The p_val is 0.07, which is greater than 0.05, so the condition p_val < 0.05 is False.

2. Step 2: Determine which branch runs

   Since the condition is False, the else block runs, printing 'Fail to reject null hypothesis'.

3. Final Answer:

   Fail to reject null hypothesis -> Option D

4. Quick Check:

   p_val = 0.07 > 0.05 means fail to reject [OK]

Hint: p_val > 0.05 means fail to reject null [OK]

Common Mistakes:

• Confusing greater than with less than
• Assuming equality means reject
• Ignoring else branch

4. This code is intended to print 'Significant' if p-value is less than 0.05, but it always prints 'Not Significant'. What is the error?

p_val = 0.03
if p_val > 0.05:
    print('Significant')
else:
    print('Not Significant')

medium

A. The comparison operator should be '<' instead of '>'

B. The variable p_val is not defined

C. The print statements are reversed

D. The p_val value is incorrect

Solution

1. Step 1: Analyze the condition logic

   The code checks if p_val > 0.05 to print 'Significant', but significance means p_val < 0.05.

2. Step 2: Identify the fix

   Changing the operator to < will correctly print 'Significant' when p_val is 0.03.

3. Final Answer:

   The comparison operator should be '<' instead of '>' -> Option A

4. Quick Check:

   Use < 0.05 for significance check [OK]

Hint: Use '<' to check p-value significance [OK]

Common Mistakes:

• Using '>' instead of '<' for significance
• Swapping print statements without fixing condition
• Assuming variable is undefined

5. You run a test and get a p-value of 0.049. You want to be very strict and use a significance level of 0.01. What should you conclude?

hard

A. Reject the null hypothesis because 0.049 < 0.05

B. Fail to reject the null hypothesis because 0.049 > 0.01

C. Reject the null hypothesis because 0.049 < 0.01

D. Fail to reject the null hypothesis because 0.049 > 0.05

Solution

1. Step 1: Understand significance level

   The significance level is set to 0.01, meaning p-values must be less than 0.01 to reject the null hypothesis.

2. Step 2: Compare p-value to significance level

   The p-value 0.049 is greater than 0.01, so we fail to reject the null hypothesis at this stricter level.

3. Final Answer:

   Fail to reject the null hypothesis because 0.049 > 0.01 -> Option B

4. Quick Check:

   p-value > significance level means fail to reject [OK]

Hint: Compare p-value to chosen significance level carefully [OK]

Common Mistakes:

• Using default 0.05 without checking chosen level
• Confusing less than and greater than
• Ignoring stricter significance level