P-values and significance in Data Analysis Python - Step-by-Step Execution

Concept Flow - P-values and significance

Start with null hypothesis

↓

Collect sample data

↓

Calculate test statistic

↓

Calculate P-value

↓

Compare P-value to significance level (alpha)

↓

Reject H0

↓

Conclude effect

This flow shows how we start with a hypothesis, collect data, calculate a P-value, and decide if the result is significant by comparing to a threshold.

Execution Sample

Data Analysis Python

import scipy.stats as stats

# Sample data
sample = [5, 7, 8, 6, 9]

# Test if mean equals 6
stat, p = stats.ttest_1samp(sample, 6)
print(f"P-value: {p:.4f}")

This code calculates the P-value for a t-test checking if the sample mean differs from 6.

Execution Table

Step	Action	Calculation	Result
1	Calculate sample mean	mean = (5+7+8+6+9)/5	mean = 7.0
2	Calculate sample std deviation	std ≈ 1.58	std ≈ 1.58
3	Calculate t-statistic	t = (7.0 - 6) / (1.58 / sqrt(5))	t ≈ 1.41
4	Calculate degrees of freedom	df = 5 - 1	df = 4
5	Calculate P-value (two-tailed)	p = 2 * (1 - CDF_t(1.41, 4))	p ≈ 0.23
6	Compare P-value to alpha=0.05	0.23 > 0.05	Fail to reject null hypothesis

💡 P-value 0.23 is greater than significance level 0.05, so we fail to reject the null hypothesis.

Variable Tracker

Variable	Start	After Step 1	After Step 2	After Step 3	After Step 4	After Step 5	Final
mean	undefined	7.0	7.0	7.0	7.0	7.0	7.0
std	undefined	undefined	1.58	1.58	1.58	1.58	1.58
t	undefined	undefined	undefined	1.41	1.41	1.41	1.41
df	undefined	undefined	undefined	undefined	4	4	4
p	undefined	undefined	undefined	undefined	undefined	0.23	0.23

Key Moments - 3 Insights

Why do we compare the P-value to the significance level (alpha)?

Because the P-value tells us how likely the observed data is if the null hypothesis is true. If it's less than alpha (like 0.05), it means the data is unlikely under the null, so we reject it. See execution_table step 6.

Does a high P-value prove the null hypothesis is true?

No, a high P-value means we do not have strong evidence against the null hypothesis, but it does not prove it is true. See execution_table step 6 where we 'fail to reject' but do not accept the null.

Why do we use a two-tailed test here?

Because we are testing if the mean is different from 6 in either direction (higher or lower). This doubles the tail probability in the P-value calculation. See execution_table step 5.

Visual Quiz - 3 Questions

Look at the execution table, what is the calculated t-statistic at step 3?

A 1.41

B 0.05

C 4

D 7.0

💡 Hint

Check the 'Result' column at step 3 in the execution_table.

At which step does the P-value get compared to the significance level?

A Step 4

B Step 5

C Step 6

D Step 2

💡 Hint

Look for the step mentioning comparison to alpha=0.05 in the execution_table.

If the sample mean was 8 instead of 7, how would the P-value change?

A It would increase

B It would decrease

C It would stay the same

D It would become zero

💡 Hint

Higher difference from null mean increases t-statistic, lowering P-value (see variable_tracker for mean and p).

Concept Snapshot

P-values measure how likely data is if the null hypothesis is true.
Calculate test statistic (like t), then P-value.
Compare P-value to significance level (alpha, e.g., 0.05).
If P-value < alpha, reject null hypothesis (significant).
If P-value >= alpha, fail to reject null (not significant).
Two-tailed tests check for difference in both directions.

Full Transcript

We start with a null hypothesis that the mean equals a value (6 here). We collect sample data and calculate the sample mean and standard deviation. Using these, we compute the t-statistic, which measures how far the sample mean is from the null mean in units of standard error. We find the degrees of freedom (sample size minus one). Then, we calculate the P-value, which tells us the probability of seeing a t-statistic as extreme as ours if the null hypothesis is true. We compare this P-value to a chosen significance level (alpha), usually 0.05. If the P-value is less than alpha, we reject the null hypothesis, concluding the sample mean is significantly different. If not, we fail to reject the null, meaning we do not have strong evidence against it. This process helps us decide if our data shows a meaningful effect or difference.