Chi-squared test in Data Analysis Python

Introduction

The Chi-squared test helps us check if two things are related or just happen by chance.

You want to see if a new teaching method affects pass rates in a class.

You want to check if people prefer different flavors of ice cream by age group.

You want to find out if a medicine works differently for men and women.

You want to test if a website layout affects user clicks based on device type.

Syntax

Data Analysis Python

from scipy.stats import chi2_contingency

chi2, p, dof, expected = chi2_contingency(observed_table)

observed_table is a 2D list or array showing counts for categories.

The function returns the test statistic, p-value, degrees of freedom, and expected counts.

Examples

Basic example with a 2x2 table of counts.

Data Analysis Python

from scipy.stats import chi2_contingency

observed = [[10, 20], [20, 40]]
chi2, p, dof, expected = chi2_contingency(observed)
print(f"p-value: {p}")

Example with a 2x3 table to test independence across more categories.

Data Analysis Python

from scipy.stats import chi2_contingency

observed = [[15, 25, 30], [10, 20, 40]]
chi2, p, dof, expected = chi2_contingency(observed)
print(f"Chi2: {chi2}, p-value: {p}")

Sample Program

This program tests if liking flavor A or B depends on gender using a Chi-squared test.

Data Analysis Python

from scipy.stats import chi2_contingency

# Observed counts of people liking two flavors by gender
observed = [
    [30, 10],  # Men: like flavor A, like flavor B
    [20, 40]   # Women: like flavor A, like flavor B
]

chi2, p, dof, expected = chi2_contingency(observed)

print(f"Chi-squared statistic: {chi2:.2f}")
print(f"Degrees of freedom: {dof}")
print(f"p-value: {p:.4f}")
print("Expected frequencies:")
for row in expected:
    print([round(x, 2) for x in row])

Important Notes

A small p-value (usually less than 0.05) means the categories are likely related.

The test needs counts, not percentages or averages.

Make sure the expected counts are not too small for reliable results.

Summary

The Chi-squared test checks if two categories are related or independent.

Use it with count data arranged in a table.

Look at the p-value to decide if the relationship is significant.

Practice

1. What does the Chi-squared test primarily check in data analysis?

easy

A. If two numerical variables have a linear relationship

B. If two categorical variables are related or independent

C. If a dataset follows a normal distribution

D. If the mean of a dataset is greater than zero

Solution

1. Step 1: Understand the purpose of the Chi-squared test

   The Chi-squared test is used to check relationships between categorical variables, not numerical data.

2. Step 2: Identify the correct relationship tested

   It tests if two categories are independent or related based on observed counts.

3. Final Answer:

   If two categorical variables are related or independent -> Option B

4. Quick Check:

   Chi-squared tests category relationships [OK]

Hint: Chi-squared tests category links, not means or correlations [OK]

Common Mistakes:

• Confusing Chi-squared with correlation tests
• Thinking it tests numerical data relationships
• Assuming it checks data distribution shape

2. Which Python library and function is commonly used to perform a Chi-squared test on a contingency table?

easy

A. scipy.stats.chi2_contingency

B. numpy.chisquare_test

C. pandas.chi_squared

D. matplotlib.chi2_test

Solution

1. Step 1: Recall common Python libraries for statistics

   Scipy is the standard library for statistical tests in Python.

2. Step 2: Identify the correct function for Chi-squared test

   The function to perform Chi-squared test on contingency tables is chi2_contingency in scipy.stats.

3. Final Answer:

   scipy.stats.chi2_contingency -> Option A

4. Quick Check:

   Use scipy.stats.chi2_contingency for Chi-squared [OK]

Hint: Remember: scipy.stats has chi2_contingency for Chi-squared [OK]

Common Mistakes:

• Using numpy or pandas which don't have this function
• Confusing function names or libraries
• Trying to use plotting libraries for tests

3. What will be the output of this Python code snippet?

import numpy as np
from scipy.stats import chi2_contingency

observed = np.array([[10, 20], [20, 40]])
chi2, p, dof, expected = chi2_contingency(observed, correction=False)
print('{:.3f}'.format(p))

medium

A. 1.000

B. 0.000

C. 0.500

D. 0.250

Solution

1. Step 1: Understand the observed data and test

   The observed table has proportional counts (10:20 and 20:40), suggesting independence.

2. Step 2: Interpret the p-value from chi2_contingency

   The p-value is approximately 0.5, indicating no significant relationship; the test finds data consistent with independence.

3. Final Answer:

   0.500 -> Option C

4. Quick Check:

   Proportional counts give moderate p-value around 0.5 [OK]

Hint: Proportional tables yield moderate p-values near 0.5 [OK]

Common Mistakes:

• Assuming low p-value means independence
• Confusing chi2 statistic with p-value
• Rounding errors changing p-value interpretation

4. Identify the error in this code that tries to perform a Chi-squared test:

import scipy.stats as stats
observed = [[15, 25], [35, 25]]
chi2, p = stats.chi2_contingency(observed)
print(p)

medium

A. Incorrect unpacking of chi2_contingency output

B. Missing import of numpy for array input

C. Observed data must be a pandas DataFrame

D. No error; code runs correctly

Solution

1. Step 1: Check the output of chi2_contingency

   The function returns four values: chi2, p, dof, expected.

2. Step 2: Verify unpacking matches returned values

   The code unpacks only two values, causing a ValueError due to mismatch.

3. Final Answer:

   Incorrect unpacking of chi2_contingency output -> Option A

4. Quick Check:

   chi2_contingency returns 4 values, unpack all [OK]

Hint: Unpack all 4 outputs from chi2_contingency [OK]

Common Mistakes:

• Unpacking fewer than 4 values
• Assuming input must be numpy array
• Thinking pandas DataFrame is required

5. You have a 3x3 contingency table of observed counts for three categories of two variables. After running chi2_contingency, you get a p-value of 0.03. What is the best interpretation if your significance level is 0.05?

hard

A. The variables are independent

B. You need a larger sample size to decide

C. The test is inconclusive; p-value is too high

D. There is a significant relationship between the variables

Solution

1. Step 1: Compare p-value with significance level

   The p-value 0.03 is less than 0.05, indicating statistical significance.

2. Step 2: Interpret significance in context of independence

   A significant result means we reject the null hypothesis of independence; variables are related.

3. Final Answer:

   There is a significant relationship between the variables -> Option D

4. Quick Check:

   p-value < 0.05 means reject independence [OK]

Hint: p-value < alpha means variables related, not independent [OK]

Common Mistakes:

• Thinking p-value < 0.05 means independence
• Ignoring significance level in decision
• Assuming inconclusive without checking p-value