Chi-squared test in Data Analysis Python - Mini Project: Build & Apply

Chi-squared Test for Independence

📖 Scenario: You work in a marketing team. You want to check if customer preference for two different products depends on their age group. You have collected data on how many customers in each age group prefer each product.

🎯 Goal: Build a Python program that uses a chi-squared test to check if product preference depends on age group.

📋 What You'll Learn

Create a contingency table as a dictionary of dictionaries with exact counts

Create a variable for the significance level

Use scipy.stats.chi2_contingency to perform the chi-squared test

Print the chi-squared statistic and p-value

💡 Why This Matters

🌍 Real World

Chi-squared tests are used in marketing, medicine, and social sciences to check if two categorical variables are related.

💼 Career

Data analysts and scientists use chi-squared tests to analyze survey data and customer preferences to make informed decisions.

1

Create the contingency table data

Create a dictionary called preference_data with keys as age groups: '18-25', '26-35', and '36-45'. Each key should map to another dictionary with keys 'Product A' and 'Product B' and these exact counts: '18-25': {'Product A': 30, 'Product B': 20}, '26-35': {'Product A': 25, 'Product B': 25}, '36-45': {'Product A': 20, 'Product B': 30}.

Data Analysis Python

# Create the preference_data dictionary with exact counts
# Your code here

Hint

Think of preference_data as a table with age groups as rows and product preferences as columns.

2

Set the significance level

Create a variable called alpha and set it to 0.05 to represent the significance level for the chi-squared test.

Data Analysis Python

preference_data = {
    '18-25': {'Product A': 30, 'Product B': 20},
    '26-35': {'Product A': 25, 'Product B': 25},
    '36-45': {'Product A': 20, 'Product B': 30}
}
# Set the significance level alpha
# Your code here

Hint

The significance level alpha is usually set to 0.05 for many tests.

3

Perform the chi-squared test

Import chi2_contingency from scipy.stats. Convert preference_data into a 2D list called table with rows for each age group and columns for 'Product A' and 'Product B' in that order. Use chi2_contingency(table) to get the chi-squared statistic and p-value. Store them in variables chi2 and p.

Data Analysis Python

preference_data = {
    '18-25': {'Product A': 30, 'Product B': 20},
    '26-35': {'Product A': 25, 'Product B': 25},
    '36-45': {'Product A': 20, 'Product B': 30}
}

alpha = 0.05

# Import chi2_contingency and create the table list
# Perform the chi-squared test and store results in chi2 and p
# Your code here

Hint

Remember to import chi2_contingency before using it. The table must be a list of lists with counts in the right order.

4

Print the chi-squared test results

Print the chi-squared statistic with the text Chi-squared statistic: followed by the value of chi2. Then print the p-value with the text p-value: followed by the value of p. Use print statements.

Data Analysis Python

from scipy.stats import chi2_contingency

preference_data = {
    '18-25': {'Product A': 30, 'Product B': 20},
    '26-35': {'Product A': 25, 'Product B': 25},
    '36-45': {'Product A': 20, 'Product B': 30}
}

alpha = 0.05

table = [
    [preference_data['18-25']['Product A'], preference_data['18-25']['Product B']],
    [preference_data['26-35']['Product A'], preference_data['26-35']['Product B']],
    [preference_data['36-45']['Product A'], preference_data['36-45']['Product B']]
]

chi2, p, dof, expected = chi2_contingency(table)

# Print the chi-squared statistic and p-value
# Your code here

Hint

Use print(f"Chi-squared statistic: {chi2}") and similarly for p-value.

Practice

1. What does the Chi-squared test primarily check in data analysis?

easy

A. If two numerical variables have a linear relationship

B. If two categorical variables are related or independent

C. If a dataset follows a normal distribution

D. If the mean of a dataset is greater than zero

Solution

1. Step 1: Understand the purpose of the Chi-squared test

   The Chi-squared test is used to check relationships between categorical variables, not numerical data.

2. Step 2: Identify the correct relationship tested

   It tests if two categories are independent or related based on observed counts.

3. Final Answer:

   If two categorical variables are related or independent -> Option B

4. Quick Check:

   Chi-squared tests category relationships [OK]

Hint: Chi-squared tests category links, not means or correlations [OK]

Common Mistakes:

• Confusing Chi-squared with correlation tests
• Thinking it tests numerical data relationships
• Assuming it checks data distribution shape

2. Which Python library and function is commonly used to perform a Chi-squared test on a contingency table?

easy

A. scipy.stats.chi2_contingency

B. numpy.chisquare_test

C. pandas.chi_squared

D. matplotlib.chi2_test

Solution

1. Step 1: Recall common Python libraries for statistics

   Scipy is the standard library for statistical tests in Python.

2. Step 2: Identify the correct function for Chi-squared test

   The function to perform Chi-squared test on contingency tables is chi2_contingency in scipy.stats.

3. Final Answer:

   scipy.stats.chi2_contingency -> Option A

4. Quick Check:

   Use scipy.stats.chi2_contingency for Chi-squared [OK]

Hint: Remember: scipy.stats has chi2_contingency for Chi-squared [OK]

Common Mistakes:

• Using numpy or pandas which don't have this function
• Confusing function names or libraries
• Trying to use plotting libraries for tests

3. What will be the output of this Python code snippet?

import numpy as np
from scipy.stats import chi2_contingency

observed = np.array([[10, 20], [20, 40]])
chi2, p, dof, expected = chi2_contingency(observed, correction=False)
print('{:.3f}'.format(p))

medium

A. 1.000

B. 0.000

C. 0.500

D. 0.250

Solution

1. Step 1: Understand the observed data and test

   The observed table has proportional counts (10:20 and 20:40), suggesting independence.

2. Step 2: Interpret the p-value from chi2_contingency

   The p-value is approximately 0.5, indicating no significant relationship; the test finds data consistent with independence.

3. Final Answer:

   0.500 -> Option C

4. Quick Check:

   Proportional counts give moderate p-value around 0.5 [OK]

Hint: Proportional tables yield moderate p-values near 0.5 [OK]

Common Mistakes:

• Assuming low p-value means independence
• Confusing chi2 statistic with p-value
• Rounding errors changing p-value interpretation

4. Identify the error in this code that tries to perform a Chi-squared test:

import scipy.stats as stats
observed = [[15, 25], [35, 25]]
chi2, p = stats.chi2_contingency(observed)
print(p)

medium

A. Incorrect unpacking of chi2_contingency output

B. Missing import of numpy for array input

C. Observed data must be a pandas DataFrame

D. No error; code runs correctly

Solution

1. Step 1: Check the output of chi2_contingency

   The function returns four values: chi2, p, dof, expected.

2. Step 2: Verify unpacking matches returned values

   The code unpacks only two values, causing a ValueError due to mismatch.

3. Final Answer:

   Incorrect unpacking of chi2_contingency output -> Option A

4. Quick Check:

   chi2_contingency returns 4 values, unpack all [OK]

Hint: Unpack all 4 outputs from chi2_contingency [OK]

Common Mistakes:

• Unpacking fewer than 4 values
• Assuming input must be numpy array
• Thinking pandas DataFrame is required

5. You have a 3x3 contingency table of observed counts for three categories of two variables. After running chi2_contingency, you get a p-value of 0.03. What is the best interpretation if your significance level is 0.05?

hard

A. The variables are independent

B. You need a larger sample size to decide

C. The test is inconclusive; p-value is too high

D. There is a significant relationship between the variables

Solution

1. Step 1: Compare p-value with significance level

   The p-value 0.03 is less than 0.05, indicating statistical significance.

2. Step 2: Interpret significance in context of independence

   A significant result means we reject the null hypothesis of independence; variables are related.

3. Final Answer:

   There is a significant relationship between the variables -> Option D

4. Quick Check:

   p-value < 0.05 means reject independence [OK]

Hint: p-value < alpha means variables related, not independent [OK]

Common Mistakes:

• Thinking p-value < 0.05 means independence
• Ignoring significance level in decision
• Assuming inconclusive without checking p-value