Confidence intervals on parameters in SciPy - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Calculate 95% confidence interval for mean with scipy

Given the sample data below, what is the 95% confidence interval for the mean using scipy.stats.t.interval?

SciPy

import numpy as np
from scipy import stats

data = np.array([5, 7, 8, 6, 9, 10, 7, 6])

mean = np.mean(data)
sem = stats.sem(data)
conf_int = stats.t.interval(0.95, len(data)-1, loc=mean, scale=sem)
print(conf_int)

A (5.35, 8.90)

B (5.50, 8.75)

C (5.60, 8.60)

D (5.10, 9.20)

💡 Hint

Use stats.sem to get the standard error of the mean, then use stats.t.interval with degrees of freedom = sample size - 1.

The 95% confidence interval for the mean is calculated using the t-distribution because the sample size is small. The standard error of the mean (SEM) is computed, then stats.t.interval returns the interval around the sample mean.

❓ data_output

intermediate

Interpret confidence interval output for regression coefficient

After fitting a linear regression, you get the 95% confidence interval for the slope as (1.2, 3.4). What does this interval mean?

A The true slope is between 1.2 and 3.4 with 95% confidence.

B 95% of the data points fall between 1.2 and 3.4.

C The slope will be exactly 2.3 in 95% of future samples.

D The slope is either 1.2 or 3.4 with 95% probability.

💡 Hint

Confidence intervals estimate where the true parameter lies with a certain confidence level.

The confidence interval means we are 95% confident that the true slope parameter lies between 1.2 and 3.4. It does not describe data points or probabilities of exact values.

🔧 Debug

advanced

Identify error in confidence interval calculation code

What error will this code raise?

import numpy as np
from scipy import stats

data = np.array([2, 4, 6, 8, 10])
mean = np.mean(data)
sem = stats.sem(data)
conf_int = stats.t.interval(0.95, len(data), loc=mean, scale=sem)
print(conf_int)

SciPy

import numpy as np
from scipy import stats

data = np.array([2, 4, 6, 8, 10])
mean = np.mean(data)
sem = stats.sem(data)
conf_int = stats.t.interval(0.95, len(data), loc=mean, scale=sem)
print(conf_int)

A IndexError: index out of range

B ValueError: degrees of freedom <= 0 for slice

C No error, prints confidence interval

D TypeError: unsupported operand type(s)

💡 Hint

Check the degrees of freedom parameter for stats.t.interval.

The degrees of freedom should be sample size minus 1. Using len(data) causes degrees of freedom to be 5, which is valid, so no error is raised. The error would occur if degrees of freedom were 0 or negative.

🚀 Application

advanced

Calculate confidence interval for variance using chi-square distribution

You have a sample variance of 4.0 from 10 observations. Using the chi-square distribution, what is the 95% confidence interval for the population variance?

SciPy

import scipy.stats as stats

n = 10
sample_var = 4.0
alpha = 0.05

lower = (n - 1) * sample_var / stats.chi2.ppf(1 - alpha/2, n - 1)
upper = (n - 1) * sample_var / stats.chi2.ppf(alpha/2, n - 1)
print((lower, upper))

A (2.10, 10.50)

B (1.20, 13.00)

C (1.80, 11.20)

D (1.56, 12.03)

💡 Hint

Use chi-square percent point function (ppf) with degrees of freedom = n - 1.

The confidence interval for variance uses chi-square distribution quantiles. The formula is ((n-1)*s² / chi2_upper, (n-1)*s² / chi2_lower).

🧠 Conceptual

expert

Why use t-distribution for confidence intervals on small samples?

Why is the t-distribution preferred over the normal distribution when calculating confidence intervals for the mean with small sample sizes?

A Because the normal distribution cannot be used for any sample size below 30.

B Because the t-distribution has heavier tails to model outliers better.

C Because the sample variance is unknown and estimated, increasing uncertainty.

D Because the t-distribution is symmetric while the normal distribution is not.

💡 Hint

Think about what changes when the sample size is small and variance is estimated.

When the sample size is small, the population variance is unknown and must be estimated from the sample. This adds uncertainty, so the t-distribution, which accounts for this extra variability, is used instead of the normal distribution.

Practice

1. What does a confidence interval represent in statistics?

easy

A. A range of values likely containing the true parameter

B. The exact value of the parameter

C. The average of the sample data

D. The maximum value observed in the data

Solution

1. Step 1: Understand the meaning of confidence interval

   A confidence interval gives a range where the true parameter is likely to be found, not a single exact value.

2. Step 2: Compare options with definition

   Only A range of values likely containing the true parameter correctly describes this range; others describe different concepts.

3. Final Answer:

   A range of values likely containing the true parameter -> Option A

4. Quick Check:

   Confidence interval = range of likely parameter values [OK]

Hint: Confidence interval = range, not exact value [OK]

Common Mistakes:

• Thinking it gives exact parameter value
• Confusing with sample mean
• Assuming it shows data maximum

2. Which of the following is the correct way to import the function to calculate confidence intervals from scipy?

easy

A. from scipy.stats import t

B. import scipy.confidence as conf

C. from scipy import confidence_interval

D. import scipy.stats.confidence

Solution

1. Step 1: Recall scipy.stats module usage

   The t-distribution and its interval function are in scipy.stats, imported as 'from scipy.stats import t'.

2. Step 2: Check other options

   Other imports do not exist or are incorrect syntax.

3. Final Answer:

   from scipy.stats import t -> Option A

4. Quick Check:

   Correct import for t interval = from scipy.stats import t [OK]

Hint: Use 'from scipy.stats import t' for confidence intervals [OK]

Common Mistakes:

• Trying to import non-existent modules
• Using wrong import syntax
• Confusing function location

3. What is the output of the following code?

import numpy as np
from scipy.stats import t

data = np.array([5, 7, 8, 6, 9])
mean = np.mean(data)
se = np.std(data, ddof=1) / np.sqrt(len(data))
interval = t.interval(0.95, len(data)-1, loc=mean, scale=se)
print(tuple(round(x, 2) for x in interval))

medium

A. (5.00, 9.00)

B. (4.50, 9.30)

C. (6.00, 7.00)

D. (5.04, 8.96)

Solution

1. Step 1: Calculate mean and standard error

   Mean = (5+7+8+6+9)/5 = 7.0; sample std dev ≈ 1.58; SE = 1.58 / sqrt(5) ≈ 0.71.

2. Step 2: Calculate 95% confidence interval using t-distribution

   Degrees of freedom = 4; t critical ≈ 2.776; interval = mean ± t * SE = 7.0 ± 2.776*0.71 ≈ (5.04, 8.96).

3. Final Answer:

   (5.04, 8.96) -> Option D

4. Quick Check:

   Mean ± t*SE = (5.04, 8.96) [OK]

Hint: Calculate mean, SE, then apply t.interval [OK]

Common Mistakes:

• Using population std dev instead of sample
• Wrong degrees of freedom
• Rounding errors

4. Identify the error in this code snippet for calculating a 90% confidence interval:

from scipy.stats import t
sample_mean = 10
sample_std = 2
n = 25
se = sample_std / n
interval = t.interval(0.90, n-1, loc=sample_mean, scale=se)
print(interval)

medium

A. Degrees of freedom should be n, not n-1

B. Wrong confidence level value

C. Standard error calculation is incorrect

D. t.interval function does not exist

Solution

1. Step 1: Check standard error calculation

   Standard error should be sample_std divided by sqrt(n), not by n.

2. Step 2: Verify other parts

   Confidence level 0.90 and degrees of freedom n-1 are correct; t.interval exists.

3. Final Answer:

   Standard error calculation is incorrect -> Option C

4. Quick Check:

   SE = std / sqrt(n), not std / n [OK]

Hint: SE = std / sqrt(n), not std / n [OK]

Common Mistakes:

• Dividing std by n instead of sqrt(n)
• Confusing degrees of freedom
• Using wrong confidence level format

5. You have a dataset with 100 measurements and want a 99% confidence interval for the mean. Which code correctly computes it using scipy?

hard

A. from scipy.stats import t
import numpy as np
data = np.random.randn(100)
mean = np.mean(data)
se = np.std(data) / 100
interval = t.interval(0.99, 100, loc=mean, scale=se)
print(interval)

B. from scipy.stats import t
import numpy as np
data = np.random.randn(100)
mean = np.mean(data)
se = np.std(data, ddof=1) / np.sqrt(100)
interval = t.interval(0.99, 99, loc=mean, scale=se)
print(interval)

C. from scipy.stats import t
import numpy as np
data = np.random.randn(100)
mean = np.mean(data)
se = np.std(data, ddof=1) / np.sqrt(100)
interval = t.interval(0.95, 99, loc=mean, scale=se)
print(interval)

D. from scipy.stats import t
import numpy as np
data = np.random.randn(100)
mean = np.mean(data)
se = np.std(data, ddof=1) / 100
interval = t.interval(0.99, 99, loc=mean, scale=se)
print(interval)

Solution

1. Step 1: Check standard error calculation

   Standard error must be sample std dev with ddof=1 divided by sqrt(n), which is 100 here.

2. Step 2: Check confidence level and degrees of freedom

   99% confidence means 0.99; degrees of freedom = n-1 = 99.

3. Step 3: Verify code correctness

   from scipy.stats import t import numpy as np data = np.random.randn(100) mean = np.mean(data) se = np.std(data, ddof=1) / np.sqrt(100) interval = t.interval(0.99, 99, loc=mean, scale=se) print(interval) correctly uses ddof=1, sqrt(100), 0.99 confidence, and 99 degrees of freedom.

4. Final Answer:

   The code with ddof=1, /np.sqrt(100), 0.99 confidence, df=99 -> Option B

5. Quick Check:

   Use ddof=1, sqrt(n), 0.99 confidence, df=n-1 [OK]

Hint: Use ddof=1 and sqrt(n) for SE; df = n-1 [OK]

Common Mistakes:

• Using population std dev (ddof=0)
• Dividing std by n instead of sqrt(n)
• Wrong confidence level or degrees of freedom