Cross-tabulation with crosstab() in Data Analysis Python - Time & Space Complexity

Time Complexity: Cross-tabulation with crosstab()

O(n)

Understanding Time Complexity

We want to understand how the time to create a cross-tabulation table grows as the data size increases.

Specifically, how does the crosstab() function handle larger datasets?

Scenario Under Consideration

Analyze the time complexity of the following code snippet.

import pandas as pd

data = pd.DataFrame({
    'Category': ['A', 'B', 'A', 'C', 'B', 'A'],
    'Type': ['X', 'Y', 'X', 'Y', 'X', 'Y']
})

result = pd.crosstab(data['Category'], data['Type'])

This code creates a table counting how many times each Category and Type pair appears.

Identify Repeating Operations

Identify the loops, recursion, array traversals that repeat.

• Primary operation: Scanning each row in the data once to count pairs.
• How many times: Exactly once per row, so n times for n rows.

How Execution Grows With Input

As the number of rows grows, the function counts pairs by checking each row once.

Input Size (n)	Approx. Operations
10	About 10 checks
100	About 100 checks
1000	About 1000 checks

Pattern observation: The work grows directly with the number of rows, so doubling rows doubles work.

Final Time Complexity

Time Complexity: O(n)

This means the time to create the crosstab grows in a straight line with the number of rows.

Common Mistake

[X] Wrong: "crosstab() checks every possible pair of categories and types, so it takes n squared time."

[OK] Correct: Actually, crosstab() just scans each row once and updates counts, it does not compare all pairs against each other.

Interview Connect

Knowing how crosstab() scales helps you explain data aggregation efficiency in interviews.

It shows you understand how counting operations relate to data size, a key skill in data science.

Self-Check

What if we added a third column to group by? How would the time complexity change?

Practice

1. What does the pd.crosstab() function do in data analysis?

easy

A. It sorts data in ascending order.

B. It counts how often categories from two variables appear together.

C. It calculates the average of numerical columns.

D. It removes missing values from the dataset.

Solution

1. Step 1: Understand the purpose of crosstab()

   The function pd.crosstab() is used to count occurrences of combinations of categories from two variables.

2. Step 2: Compare with other options

   Sorting, averaging, and removing missing values are different functions, not related to crosstab.

3. Final Answer:

   It counts how often categories from two variables appear together. -> Option B

4. Quick Check:

   Cross-tabulation = counts of category pairs [OK]

Hint: Remember: crosstab counts category pairs, not calculations [OK]

Common Mistakes:

• Thinking crosstab sorts data
• Confusing crosstab with aggregation functions
• Assuming crosstab cleans data

2. Which of the following is the correct syntax to create a cross-tabulation table using pandas?

easy

A. pd.crosstab(index=df['AgeGroup'], columns=df['Gender'])

B. pd.crosstab(df.Gender, columns='AgeGroup')

C. pd.crosstab(columns=df['Gender'], index=df['AgeGroup'])

D. pd.crosstab(df['Gender'], df['AgeGroup'])

Solution

1. Step 1: Recall the correct parameter order for crosstab()

   The function pd.crosstab() takes the first argument as the row variable and the second as the column variable, both as Series or arrays.

2. Step 2: Check each option

   pd.crosstab(df['Gender'], df['AgeGroup']) correctly passes two Series without naming parameters. pd.crosstab(index=df['AgeGroup'], columns=df['Gender']) uses named parameters with row and column variables reversed. pd.crosstab(df.Gender, columns='AgeGroup') passes a string literal instead of a Series for the columns parameter. pd.crosstab(columns=df['Gender'], index=df['AgeGroup']) swaps index and columns incorrectly.

3. Final Answer:

   pd.crosstab(df['Gender'], df['AgeGroup']) -> Option D

4. Quick Check:

   Use positional args: rows, columns [OK]

Hint: Use positional args: first is rows, second is columns [OK]

Common Mistakes:

• Swapping index and columns parameters
• Mixing positional and named arguments incorrectly
• Passing DataFrame instead of Series

3. Given the DataFrame:

import pandas as pd
data = {'Color': ['Red', 'Blue', 'Red', 'Green', 'Blue', 'Green'],
        'Shape': ['Circle', 'Square', 'Square', 'Circle', 'Circle', 'Square']}
df = pd.DataFrame(data)
print(pd.crosstab(df['Color'], df['Shape']))

What is the output?

medium

A. Shape Circle Square
Color
Blue 1 1
Green 1 1
Red 1 1

B. Shape Circle Square
Color
Blue 2 0
Green 0 2
Red 1 1

C. Shape Circle Square
Color
Blue 1 0
Green 1 1
Red 1 1

D. Shape Circle Square
Color
Blue 1 1
Green 1 0
Red 1 1

Solution

1. Step 1: Count occurrences of each Color-Shape pair

   From the data:
   - Blue: Circle(1), Square(1)
   - Green: Circle(1), Square(1)
   - Red: Circle(1), Square(1)

2. Step 2: Construct the crosstab table

   Each color has one Circle and one Square, so the table shows 1 for each pair.

3. Final Answer:

   Shape Circle Square Color Blue 1 1 Green 1 1 Red 1 1 -> Option A

4. Quick Check:

   Count pairs correctly = Shape Circle Square Color Blue 1 1 Green 1 1 Red 1 1 [OK]

Hint: Count each category pair carefully from data [OK]

Common Mistakes:

• Miscounting pairs
• Swapping rows and columns
• Ignoring repeated categories

4. What is wrong with this code snippet?

import pandas as pd
data = {'Team': ['A', 'B', 'A', 'B'], 'Result': ['Win', 'Lose', 'Lose', 'Win']}
df = pd.DataFrame(data)
ct = pd.crosstab(index=df['Team'], columns='Result')
print(ct)

medium

A. The DataFrame is missing a column named 'Result'.

B. The index parameter should be a list, not a Series.

C. The columns parameter should be a Series, not a string.

D. The crosstab function requires numeric data only.

Solution

1. Step 1: Check the columns argument in crosstab()

   The columns argument must be a Series or array representing categories, not a string literal.

2. Step 2: Identify the error in the code

   Here, columns='Result' is a string, not df['Result']. This causes crosstab to treat it incorrectly.

3. Final Answer:

   The columns parameter should be a Series, not a string. -> Option C

4. Quick Check:

   Columns must be Series, not string [OK]

Hint: Pass Series for index and columns, not strings [OK]

Common Mistakes:

• Passing column name as string instead of Series
• Assuming crosstab needs numeric data
• Confusing index and columns parameters

5. You have a DataFrame with columns 'Department' and 'Satisfaction' (values: 'High', 'Medium', 'Low'). You want to find how many employees in each department have each satisfaction level, but only show departments with at least 3 employees total. Which code correctly does this?

hard

A. ct = pd.crosstab(df['Department'], df['Satisfaction'])
ct = ct[ct.sum(axis=1) >= 3]

B. ct = pd.crosstab(index=df['Satisfaction'], columns=df['Department'])
ct = ct[ct.sum(axis=0) >= 3]

C. ct = pd.crosstab(df['Department'], df['Satisfaction'])
ct = ct[ct.sum(axis=0) >= 3]

D. ct = pd.crosstab(index=df['Department'], columns=df['Satisfaction'])
ct = ct[ct.sum(axis=1) < 3]

Solution

1. Step 1: Create crosstab with Department as rows and Satisfaction as columns

   Use pd.crosstab(df['Department'], df['Satisfaction']) to count satisfaction levels per department.

2. Step 2: Filter departments with total employees >= 3

   Sum counts across columns (axis=1) for each department, then keep rows where sum >= 3.

3. Final Answer:

   ct = pd.crosstab(df['Department'], df['Satisfaction']) ct = ct[ct.sum(axis=1) >= 3] -> Option A

4. Quick Check:

   Filter rows by sum(axis=1) >= 3 [OK]

Hint: Sum rows (axis=1) to filter departments by total count [OK]

Common Mistakes:

• Filtering columns instead of rows
• Swapping index and columns in crosstab
• Using wrong axis in sum()