Why hypothesis testing validates claims in SciPy - Challenge Your Understanding

Challenge - 5 Problems

🧠 Conceptual

intermediate

What does the p-value represent in hypothesis testing?

In hypothesis testing, what does the p-value tell us about the claim we are testing?

A The probability that the observed data is incorrect.

B The probability that the alternative hypothesis is true given the data.

C The probability that the null hypothesis is false.

D The probability of observing the data, or something more extreme, assuming the null hypothesis is true.

💡 Hint

Think about what the p-value measures under the assumption that the null hypothesis holds.

The p-value measures how likely it is to get the observed data (or more extreme) if the null hypothesis is true. It does not directly tell us the probability that the null or alternative hypothesis is true.

❓ Predict Output

intermediate

Output of a t-test on sample data

Given the following Python code using scipy.stats, what is the output of the t-statistic and p-value?

SciPy

from scipy import stats

sample = [5, 7, 8, 6, 9, 7, 5]
# Test if the sample mean is different from 6
result = stats.ttest_1samp(sample, 6)
print(round(result.statistic, 2), round(result.pvalue, 3))

A 1.26 0.250

B 0.87 0.412

C 2.45 0.045

D 3.12 0.015

💡 Hint

Calculate the sample mean and standard deviation, then apply the t-test formula or run the code.

The t-test compares the sample mean to the population mean 6. The calculated t-statistic is about 1.26 and the p-value about 0.250, meaning no strong evidence to reject the null hypothesis.

❓ data_output

advanced

Result of a two-sample t-test on two groups

Two groups have the following data:

Group A: [12, 15, 14, 10, 13]

Group B: [8, 9, 7, 6, 10]

Using a two-sample t-test, what is the t-statistic and p-value?

SciPy

from scipy import stats

group_a = [12, 15, 14, 10, 13]
group_b = [8, 9, 7, 6, 10]
result = stats.ttest_ind(group_a, group_b)
print(round(result.statistic, 2), round(result.pvalue, 3))

A 4.31 0.002

B 2.13 0.075

C 3.58 0.010

D 1.98 0.120

💡 Hint

Calculate the means and variances of both groups, then apply the two-sample t-test or run the code.

The t-statistic is about 4.31 with a p-value of 0.002, indicating strong evidence that the two groups have different means.

❓ visualization

advanced

Interpreting a p-value distribution plot

Below is a plot showing the distribution of p-values from many hypothesis tests under the null hypothesis. What does the uniform distribution of p-values indicate?

SciPy

import matplotlib.pyplot as plt
import numpy as np

np.random.seed(0)
p_values = np.random.uniform(0, 1, 1000)
plt.hist(p_values, bins=20, edgecolor='black')
plt.title('Distribution of p-values under Null Hypothesis')
plt.xlabel('p-value')
plt.ylabel('Frequency')
plt.show()

A Most p-values cluster near zero indicating strong evidence against the null hypothesis.

B P-values are equally likely across the range 0 to 1 when the null hypothesis is true.

C P-values cluster near one indicating the null hypothesis is always true.

D P-values follow a normal distribution centered at 0.5.

💡 Hint

Think about what the p-value distribution looks like if the null hypothesis is true for all tests.

When the null hypothesis is true, p-values are uniformly distributed between 0 and 1, meaning any p-value is equally likely.

🔧 Debug

expert

Identify the error in hypothesis testing code

What error will this code raise when running a one-sample t-test?

from scipy import stats
sample = [2, 4, 6, 8]
result = stats.ttest_1samp(sample)
print(result)

A No error, prints t-statistic and p-value

B ValueError: Sample size must be greater than 1

C TypeError: ttest_1samp() missing 1 required positional argument: 'popmean'

D NameError: name 'stats' is not defined

💡 Hint

Check the required arguments for ttest_1samp function.

The function ttest_1samp requires two arguments: the sample and the population mean to test against. Omitting the population mean causes a TypeError.

Practice

1. What does a small p-value in hypothesis testing usually indicate about the claim being tested?

easy

A. The data is not enough to decide

B. The claim is definitely true

C. The claim is probably false and should be rejected

D. The claim is definitely false

Solution

1. Step 1: Understand p-value meaning

   A p-value measures how likely the observed data would occur if the claim (null hypothesis) is true.

2. Step 2: Interpret small p-value

   A small p-value means the observed data is unlikely under the claim, so we reject the claim.

3. Final Answer:

   The claim is probably false and should be rejected -> Option C

4. Quick Check:

   Small p-value = reject claim [OK]

Hint: Small p-value means reject claim, large means accept [OK]

Common Mistakes:

• Thinking small p-value proves claim true
• Confusing p-value with probability of claim
• Assuming p-value alone proves claim false

2. Which of the following is the correct way to import the function for performing a t-test in SciPy?

easy

A. from scipy.stats import ttest_ind

B. import scipy.ttest as ttest_ind

C. from scipy import ttest_ind

D. import scipy.stats.ttest_ind()

Solution

1. Step 1: Recall SciPy stats module structure

   The t-test function is inside scipy.stats and is called ttest_ind.

2. Step 2: Check correct import syntax

   Correct syntax is 'from scipy.stats import ttest_ind' to import the function directly.

3. Final Answer:

   from scipy.stats import ttest_ind -> Option A

4. Quick Check:

   Correct import = from scipy.stats import ttest_ind [OK]

Hint: Use 'from scipy.stats import ttest_ind' to import t-test [OK]

Common Mistakes:

• Using incorrect module path
• Trying to import with parentheses
• Confusing import syntax with function call

3. Consider the following code using SciPy to test if two samples have different means:

import numpy as np
from scipy.stats import ttest_ind

sample1 = np.array([5, 7, 8, 6, 9])
sample2 = np.array([10, 12, 11, 13, 14])
stat, p = ttest_ind(sample1, sample2)
print(round(p, 3))

What is the printed p-value rounded to 3 decimals?

medium

A. 0.000

B. 0.050

C. 0.001

D. 0.100

Solution

1. Step 1: Run t-test on given samples

   Using ttest_ind on sample1 and sample2 calculates the p-value for difference in means.

2. Step 2: Calculate and round p-value

   Running the code gives p approximately 0.00002, which rounds to 0.000.

3. Final Answer:

   0.000 -> Option A

4. Quick Check:

   p-value ≈ 0.000 means very strong evidence against claim [OK]

Hint: Run code or estimate p-value from sample difference [OK]

Common Mistakes:

• Rounding p-value incorrectly
• Confusing test statistic with p-value
• Assuming p-value is zero without rounding

4. The following code attempts to perform a hypothesis test but has an error:

from scipy.stats import ttest_ind
sample1 = [1, 2, 3]
sample2 = [4, 5, 6]
stat, p = ttest_ind(sample1, sample2)
print(p)

What is the error and how to fix it?

medium

A. Import statement is incorrect, should be 'import scipy.stats.ttest_ind'

B. No error; code runs correctly

C. ttest_ind requires samples to be same length

D. sample1 and sample2 must be numpy arrays, convert with np.array()

Solution

1. Step 1: Check input types for ttest_ind

   ttest_ind accepts lists or arrays; lists are valid inputs.

2. Step 2: Verify code runs without error

   Running this code prints a p-value without error, so no fix needed.

3. Final Answer:

   No error; code runs correctly -> Option B

4. Quick Check:

   Lists accepted by ttest_ind = No error; code runs correctly [OK]

Hint: Lists work fine as input to ttest_ind [OK]

Common Mistakes:

• Thinking inputs must be numpy arrays
• Assuming sample sizes must match
• Incorrect import syntax

5. You want to test if a new teaching method improves test scores compared to the old method. You collect scores from two groups and run a t-test using SciPy. The p-value is 0.07. What is the best conclusion if you use a significance level of 0.05?

hard

A. The new method definitely does not improve scores

B. Accept the claim that the new method improves scores

C. Reject the claim that the new method improves scores

D. There is not enough evidence to reject the claim

Solution

1. Step 1: Compare p-value to significance level

   The p-value 0.07 is greater than 0.05, so we do not reject the null hypothesis.

2. Step 2: Interpret result in context

   Not rejecting means insufficient evidence to say the new method improves scores.

3. Final Answer:

   There is not enough evidence to reject the claim -> Option D

4. Quick Check:

   p > 0.05 means insufficient evidence [OK]

Hint: p > 0.05 means no strong evidence to reject claim [OK]

Common Mistakes:

• Confusing 'not reject' with 'accept claim'
• Thinking p=0.07 proves claim true
• Ignoring significance level threshold