Confidence intervals on parameters in SciPy - Step-by-Step Execution

Concept Flow - Confidence intervals on parameters

Collect sample data

↓

Estimate parameter (e.g., mean)

↓

Calculate standard error

↓

Choose confidence level (e.g., 95%)

↓

Find critical value from distribution

↓

Calculate margin of error

↓

Construct confidence interval

↓

Interpret interval as plausible range for parameter

Start with data, estimate parameter and error, pick confidence level, find critical value, compute margin, then build interval.

Execution Sample

SciPy

import numpy as np
from scipy import stats

data = np.array([5, 7, 8, 6, 9])
mean = np.mean(data)
se = stats.sem(data)
ci = stats.t.interval(0.95, len(data)-1, loc=mean, scale=se)
print(ci)

Calculate 95% confidence interval for the mean of a small sample using t-distribution.

Execution Table

Step	Action	Value/Result	Explanation
1	Calculate mean	7.0	Mean of data [5,7,8,6,9] is (5+7+8+6+9)/5 = 7.0
2	Calculate standard error (se)	0.7071	Standard deviation divided by sqrt(n), se ≈ 0.7071
3	Degrees of freedom	4	Sample size 5 minus 1 = 4
4	Find t critical value	2.776	From t-table for 95% CI and df=4, two-tailed
5	Calculate margin of error	1.963	t_crit * se = 2.776 * 0.7071 ≈ 1.963
6	Construct confidence interval	(5.037, 8.963)	Mean ± margin = 7.0 ± 1.963
7	Print confidence interval	(5.037, 8.963)	Final output shows plausible range for true mean

💡 Confidence interval calculated and printed for sample mean.

Variable Tracker

Variable	Start	After Step 1	After Step 2	After Step 4	After Step 5	Final
data	[5,7,8,6,9]	[5,7,8,6,9]	[5,7,8,6,9]	[5,7,8,6,9]	[5,7,8,6,9]	[5,7,8,6,9]
mean	N/A	7.0	7.0	7.0	7.0	7.0
se	N/A	N/A	0.7071	0.7071	0.7071	0.7071
df	N/A	N/A	N/A	4	4	4
t_crit	N/A	N/A	N/A	2.776	2.776	2.776
margin	N/A	N/A	N/A	N/A	1.963	1.963
ci	N/A	N/A	N/A	N/A	N/A	(5.037, 8.963)

Key Moments - 3 Insights

Why do we use the t-distribution instead of the normal distribution here?

Because the sample size is small (n=5), and the population standard deviation is unknown, the t-distribution better accounts for extra uncertainty, as shown in step 4 where t critical value is used.

What does the margin of error represent in the confidence interval?

The margin of error (step 5) is how far above and below the sample mean we go to create the interval, reflecting uncertainty in the estimate.

Why is the degrees of freedom equal to 4?

Degrees of freedom is sample size minus one (n-1), so 5-1=4, used to find the correct t critical value in step 4.

Visual Quiz - 3 Questions

Look at the execution table, what is the value of the standard error after step 2?

A 7.0

B 0.7071

C 2.776

D 4

💡 Hint

Check the 'Value/Result' column at step 2 in the execution_table.

At which step is the margin of error calculated?

A Step 6

B Step 3

C Step 5

D Step 4

💡 Hint

Look for 'Calculate margin of error' in the 'Action' column of execution_table.

If the sample size increased, how would the margin of error change?

A It would decrease

B It would stay the same

C It would increase

D It would become zero

💡 Hint

Refer to variable_tracker and remember margin depends on standard error which decreases with larger sample size.

Concept Snapshot

Confidence intervals estimate a range for a parameter.
Calculate sample estimate (mean), standard error.
Choose confidence level (e.g., 95%).
Find critical value (t or z).
Margin = critical * standard error.
Interval = estimate ± margin.

Full Transcript

This visual execution traces how to calculate a confidence interval for a sample mean using Python and scipy. We start with sample data, calculate the mean and standard error. Because the sample is small, we use the t-distribution with degrees of freedom equal to sample size minus one. We find the critical t value for 95% confidence, then calculate the margin of error by multiplying the critical value by the standard error. Finally, we build the confidence interval by adding and subtracting the margin from the mean. The output is a range that likely contains the true population mean. Key points include why the t-distribution is used, what margin of error means, and how degrees of freedom affect the critical value.

Practice

1. What does a confidence interval represent in statistics?

easy

A. A range of values likely containing the true parameter

B. The exact value of the parameter

C. The average of the sample data

D. The maximum value observed in the data

Solution

1. Step 1: Understand the meaning of confidence interval

   A confidence interval gives a range where the true parameter is likely to be found, not a single exact value.

2. Step 2: Compare options with definition

   Only A range of values likely containing the true parameter correctly describes this range; others describe different concepts.

3. Final Answer:

   A range of values likely containing the true parameter -> Option A

4. Quick Check:

   Confidence interval = range of likely parameter values [OK]

Hint: Confidence interval = range, not exact value [OK]

Common Mistakes:

• Thinking it gives exact parameter value
• Confusing with sample mean
• Assuming it shows data maximum

2. Which of the following is the correct way to import the function to calculate confidence intervals from scipy?

easy

A. from scipy.stats import t

B. import scipy.confidence as conf

C. from scipy import confidence_interval

D. import scipy.stats.confidence

Solution

1. Step 1: Recall scipy.stats module usage

   The t-distribution and its interval function are in scipy.stats, imported as 'from scipy.stats import t'.

2. Step 2: Check other options

   Other imports do not exist or are incorrect syntax.

3. Final Answer:

   from scipy.stats import t -> Option A

4. Quick Check:

   Correct import for t interval = from scipy.stats import t [OK]

Hint: Use 'from scipy.stats import t' for confidence intervals [OK]

Common Mistakes:

• Trying to import non-existent modules
• Using wrong import syntax
• Confusing function location

3. What is the output of the following code?

import numpy as np
from scipy.stats import t

data = np.array([5, 7, 8, 6, 9])
mean = np.mean(data)
se = np.std(data, ddof=1) / np.sqrt(len(data))
interval = t.interval(0.95, len(data)-1, loc=mean, scale=se)
print(tuple(round(x, 2) for x in interval))

medium

A. (5.00, 9.00)

B. (4.50, 9.30)

C. (6.00, 7.00)

D. (5.04, 8.96)

Solution

1. Step 1: Calculate mean and standard error

   Mean = (5+7+8+6+9)/5 = 7.0; sample std dev ≈ 1.58; SE = 1.58 / sqrt(5) ≈ 0.71.

2. Step 2: Calculate 95% confidence interval using t-distribution

   Degrees of freedom = 4; t critical ≈ 2.776; interval = mean ± t * SE = 7.0 ± 2.776*0.71 ≈ (5.04, 8.96).

3. Final Answer:

   (5.04, 8.96) -> Option D

4. Quick Check:

   Mean ± t*SE = (5.04, 8.96) [OK]

Hint: Calculate mean, SE, then apply t.interval [OK]

Common Mistakes:

• Using population std dev instead of sample
• Wrong degrees of freedom
• Rounding errors

4. Identify the error in this code snippet for calculating a 90% confidence interval:

from scipy.stats import t
sample_mean = 10
sample_std = 2
n = 25
se = sample_std / n
interval = t.interval(0.90, n-1, loc=sample_mean, scale=se)
print(interval)

medium

A. Degrees of freedom should be n, not n-1

B. Wrong confidence level value

C. Standard error calculation is incorrect

D. t.interval function does not exist

Solution

1. Step 1: Check standard error calculation

   Standard error should be sample_std divided by sqrt(n), not by n.

2. Step 2: Verify other parts

   Confidence level 0.90 and degrees of freedom n-1 are correct; t.interval exists.

3. Final Answer:

   Standard error calculation is incorrect -> Option C

4. Quick Check:

   SE = std / sqrt(n), not std / n [OK]

Hint: SE = std / sqrt(n), not std / n [OK]

Common Mistakes:

• Dividing std by n instead of sqrt(n)
• Confusing degrees of freedom
• Using wrong confidence level format

5. You have a dataset with 100 measurements and want a 99% confidence interval for the mean. Which code correctly computes it using scipy?

hard

A. from scipy.stats import t
import numpy as np
data = np.random.randn(100)
mean = np.mean(data)
se = np.std(data) / 100
interval = t.interval(0.99, 100, loc=mean, scale=se)
print(interval)

B. from scipy.stats import t
import numpy as np
data = np.random.randn(100)
mean = np.mean(data)
se = np.std(data, ddof=1) / np.sqrt(100)
interval = t.interval(0.99, 99, loc=mean, scale=se)
print(interval)

C. from scipy.stats import t
import numpy as np
data = np.random.randn(100)
mean = np.mean(data)
se = np.std(data, ddof=1) / np.sqrt(100)
interval = t.interval(0.95, 99, loc=mean, scale=se)
print(interval)

D. from scipy.stats import t
import numpy as np
data = np.random.randn(100)
mean = np.mean(data)
se = np.std(data, ddof=1) / 100
interval = t.interval(0.99, 99, loc=mean, scale=se)
print(interval)

Solution

1. Step 1: Check standard error calculation

   Standard error must be sample std dev with ddof=1 divided by sqrt(n), which is 100 here.

2. Step 2: Check confidence level and degrees of freedom

   99% confidence means 0.99; degrees of freedom = n-1 = 99.

3. Step 3: Verify code correctness

   from scipy.stats import t import numpy as np data = np.random.randn(100) mean = np.mean(data) se = np.std(data, ddof=1) / np.sqrt(100) interval = t.interval(0.99, 99, loc=mean, scale=se) print(interval) correctly uses ddof=1, sqrt(100), 0.99 confidence, and 99 degrees of freedom.

4. Final Answer:

   The code with ddof=1, /np.sqrt(100), 0.99 confidence, df=99 -> Option B

5. Quick Check:

   Use ddof=1, sqrt(n), 0.99 confidence, df=n-1 [OK]

Hint: Use ddof=1 and sqrt(n) for SE; df = n-1 [OK]

Common Mistakes:

• Using population std dev (ddof=0)
• Dividing std by n instead of sqrt(n)
• Wrong confidence level or degrees of freedom