P-values and significance in Data Analysis Python - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Understanding P-value from a t-test

What is the output of the following Python code that performs a t-test on two small samples?

Data Analysis Python

from scipy.stats import ttest_ind
sample1 = [5, 7, 8, 6, 9]
sample2 = [10, 12, 11, 13, 14]
t_stat, p_value = ttest_ind(sample1, sample2)
print(round(p_value, 3))

A 0.05

B 0.002

C 0.5

D 0.1

💡 Hint

Think about how different the two samples are and what a small p-value means.

The t-test compares the means of two samples. Because the samples differ clearly, the p-value is very small, indicating strong evidence against the null hypothesis.

❓ data_output

intermediate

Interpreting significance from p-values in a DataFrame

Given the DataFrame below with p-values from multiple tests, which rows are considered statistically significant at alpha = 0.05?

Data Analysis Python

import pandas as pd
data = {'Test': ['A', 'B', 'C', 'D'], 'p_value': [0.03, 0.07, 0.001, 0.2]}
df = pd.DataFrame(data)
significant = df[df['p_value'] < 0.05]
print(significant['Test'].tolist())

A ['A', 'B', 'C']

B ['B', 'D']

C ['A', 'C']

D ['C', 'D']

💡 Hint

Remember significance means p-value less than 0.05.

Only tests A and C have p-values below 0.05, so they are significant.

🧠 Conceptual

advanced

Effect of sample size on p-value

Which statement best explains how increasing sample size affects the p-value in hypothesis testing?

A Increasing sample size decreases variability, which can lead to smaller p-values if the effect exists.

B Increasing sample size always increases the p-value, making results less significant.

C Sample size does not affect the p-value; it only depends on the observed effect size.

D Increasing sample size always leads to p-values equal to zero.

💡 Hint

Think about how more data affects the certainty of results.

Larger samples reduce random noise, making it easier to detect true effects, often lowering p-values if an effect is present.

🔧 Debug

advanced

Identifying error in p-value calculation code

What error will this code produce when trying to calculate a p-value from a t-test?

Data Analysis Python

from scipy.stats import ttest_ind
sample1 = [1, 2, 3]
sample2 = [4, 5]
t_stat, p_value = ttest_ind(sample1, sample2, equal_var=False)
print(p_value)

A IndexError due to list indexing

B ValueError due to unequal sample sizes

C TypeError because equal_var is not a valid argument

D No error; prints a valid p-value

💡 Hint

Check the documentation for ttest_ind about sample sizes and parameters.

The function supports unequal sample sizes and the equal_var parameter is valid, so no error occurs.

🚀 Application

expert

Choosing significance level for multiple tests

You run 20 independent hypothesis tests each at alpha = 0.05. What is the approximate probability of getting at least one false positive (Type I error) by chance?

A About 0.64

B About 0.95

C About 1.0

D Exactly 0.05

💡 Hint

Use the formula for the complement of no false positives: 1 - (1 - alpha)^number_of_tests.

The chance of no false positives is (1 - 0.05)^20 ≈ 0.358, so the chance of at least one false positive is about 1 - 0.358 = 0.642.