P-values and significance in Data Analysis Python - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is a p-value in hypothesis testing?

A p-value is the probability of getting results at least as extreme as the observed results, assuming the null hypothesis is true.

beginner

What does it mean if a p-value is less than 0.05?

It means the results are statistically significant, so we reject the null hypothesis at the 5% significance level.

intermediate

Why do we set a significance level (alpha) before testing?

We set alpha to decide the threshold for rejecting the null hypothesis, controlling the chance of a false positive (Type I error).

beginner

What is the null hypothesis?

The null hypothesis is a statement that there is no effect or no difference, and it is what we test against in statistical tests.

intermediate

Can a p-value tell us the size or importance of an effect?

No, a p-value only tells us about statistical significance, not the size or practical importance of the effect.

What does a p-value measure?

A The probability that the alternative hypothesis is true

B The probability that the null hypothesis is true

C The size of the effect in the data

D The probability of observing the data if the null hypothesis is true

A p-value measures how likely the observed data is assuming the null hypothesis is true.

If the p-value is 0.03 and alpha is 0.05, what should you do?

A Reject the null hypothesis

B Increase the sample size

C Fail to reject the null hypothesis

D Accept the null hypothesis

Since 0.03 < 0.05, the result is statistically significant and we reject the null hypothesis.

What is the common significance level used in many tests?

A 0.05

B 0.10

C 0.5

D 0.01

The 0.05 level is commonly used as the cutoff for significance.

Which error means rejecting a true null hypothesis?

A Type II error

B Type I error

C Sampling error

D Measurement error

Type I error is when we wrongly reject a true null hypothesis.

Does a small p-value prove the alternative hypothesis is true?

A Yes, it proves it

B Yes, with 100% certainty

C No, it only suggests evidence against the null

D No, it proves the null hypothesis

A small p-value suggests evidence against the null but does not prove the alternative hypothesis.

Explain what a p-value is and how it is used to decide significance in hypothesis testing.

Think about the chance of seeing your data if nothing is happening.

Describe the difference between statistical significance and practical importance.

A result can be statistically significant but not useful in real life.