Cross-tabulation advanced usage in Pandas - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is cross-tabulation in pandas used for?

Cross-tabulation is used to compute a simple frequency table of two or more factors. It helps to understand the relationship between categorical variables by showing counts or other statistics in a matrix format.

beginner

How can you add margins (totals) to a pandas crosstab?

Use the parameter margins=True in pd.crosstab(). This adds row and column totals to the table, helping to see overall counts.

intermediate

What does the normalize parameter do in pd.crosstab()?

The normalize parameter converts counts into proportions. You can normalize by 'index', 'columns', or 'all' to get relative frequencies instead of raw counts.

advanced

How do you include multiple aggregation functions in a pandas crosstab?

Use the aggfunc parameter with a list of functions (e.g., [np.sum, np.mean]) and provide a values argument to specify the data to aggregate.

advanced

Explain how to create a multi-index crosstab with more than two categorical variables.

Pass multiple arrays or columns to the index and/or columns parameters as lists. This creates a multi-level index in the resulting table, showing detailed breakdowns.

Which parameter in pd.crosstab() adds row and column totals?

A normalize='all'

B margins=True

C aggfunc='sum'

D dropna=False

The margins=True parameter adds totals for rows and columns in the crosstab.

What does normalize='index' do in a crosstab?

A Shows proportions across each row

B Shows proportions across each column

C Shows overall proportions

D Removes missing values

Normalizing by 'index' converts counts into proportions across each row.

How do you specify multiple aggregation functions in a crosstab?

A aggfunc=None

B aggfunc='sum,mean'

C aggfunc='multiple'

D aggfunc=[np.sum, np.mean]

You pass a list of functions like [np.sum, np.mean] to the aggfunc parameter.

To create a multi-level index in a crosstab, you should:

A Use margins=True

B Set normalize='columns'

C Pass lists of columns to index and columns parameters

D Use dropna=True

Passing multiple columns as lists to index or columns creates multi-level indexes.

Which of these is NOT a valid use of pd.crosstab()?

A Creating a scatter plot

B Aggregating numerical data with multiple functions

C Calculating frequency counts between two variables

D Normalizing counts to proportions

Cross-tabulation does not create scatter plots; it summarizes categorical data.

Describe how to use pandas crosstab to analyze the relationship between three categorical variables with totals and normalized proportions.

Think about multi-index and normalization options in crosstab.

Explain how to apply multiple aggregation functions on numerical data within a pandas crosstab.

Consider how aggfunc and values work together.

Practice

1. What does the pd.crosstab() function primarily do in pandas?

easy

A. It sorts a DataFrame by multiple columns.

B. It computes a frequency table of two or more factors.

C. It merges two DataFrames based on a key column.

D. It filters rows based on a condition.

Solution

1. Step 1: Understand the purpose of pd.crosstab()

   pd.crosstab() is used to compute a cross-tabulation, which is a frequency table showing counts of combinations of factors.

2. Step 2: Compare with other pandas functions

   Sorting, merging, and filtering are done by other functions like sort_values(), merge(), and boolean indexing, not crosstab.

3. Final Answer:

   It computes a frequency table of two or more factors. -> Option B

4. Quick Check:

   Cross-tabulation = frequency table [OK]

Hint: Remember: crosstab counts combinations of categories [OK]

Common Mistakes:

• Confusing crosstab with merge or join operations
• Thinking crosstab sorts data
• Assuming crosstab filters rows

2. Which of the following is the correct syntax to create a cross-tabulation with totals (margins) included?

easy

A. pd.crosstab(df['A'], df['B'], include_margins=True)

B. pd.crosstab(df['A'], df['B'], total=True)

C. pd.crosstab(df['A'], df['B'], add_totals=True)

D. pd.crosstab(df['A'], df['B'], margins=True)

Solution

1. Step 1: Recall the parameter for totals in pd.crosstab()

   The correct parameter to add row and column totals is margins=True.

2. Step 2: Check other options for correctness

   Parameters like total, add_totals, or include_margins do not exist in pd.crosstab.

3. Final Answer:

   pd.crosstab(df['A'], df['B'], margins=True) -> Option D

4. Quick Check:

   margins=True adds totals [OK]

Hint: Use margins=True to add totals in crosstab [OK]

Common Mistakes:

• Using incorrect parameter names for totals
• Forgetting to set margins=True
• Confusing margins with normalization

3. Given the DataFrame:

import pandas as pd
data = {'City': ['NY', 'LA', 'NY', 'LA', 'NY'], 'Category': ['A', 'A', 'B', 'B', 'A'], 'Sales': [100, 200, 150, 300, 250]}
df = pd.DataFrame(data)

What is the output of:

pd.crosstab(df['City'], df['Category'], values=df['Sales'], aggfunc='sum')

medium

A. {'A': {'LA': 200, 'NY': 350}, 'B': {'LA': 300, 'NY': 150}}

B. {'A': {'LA': 100, 'NY': 250}, 'B': {'LA': 150, 'NY': 300}}

C. {'A': {'LA': 300, 'NY': 350}, 'B': {'LA': 200, 'NY': 150}}

D. {'A': {'LA': 200, 'NY': 250}, 'B': {'LA': 300, 'NY': 150}}

Solution

1. Step 1: Group sales by City and Category with sum aggregation

   For LA and A: sales = 200; LA and B: 300; NY and A: 100 + 250 = 350; NY and B: 150.

2. Step 2: Check the options carefully

   {'A': {'LA': 200, 'NY': 250}, 'B': {'LA': 300, 'NY': 150}} matches the sums except NY and A is 250 in D, but actual sum is 100 + 250 = 350, so re-check.

3. Step 3: Recalculate sums

   NY & A: rows 0 and 4 have sales 100 and 250, total 350. So correct sums are: LA A=200, LA B=300, NY A=350, NY B=150.

4. Step 4: Match with options

   {'A': {'LA': 200, 'NY': 350}, 'B': {'LA': 300, 'NY': 150}} matches these sums exactly.

5. Final Answer:

   {'A': {'LA': 200, 'NY': 350}, 'B': {'LA': 300, 'NY': 150}} -> Option A

6. Quick Check:

   Sum sales by City and Category = {'A': {'LA': 200, 'NY': 350}, 'B': {'LA': 300, 'NY': 150}} [OK]

Hint: Sum sales by groups to match crosstab output [OK]

Common Mistakes:

• Adding wrong sales values for groups
• Confusing rows for LA and NY
• Ignoring aggfunc='sum' effect

4. Consider this code snippet:

pd.crosstab(index=df['City'], columns=df['Category'], values=df['Sales'], aggfunc='mean', margins=True)

It raises a TypeError. What is the likely cause?

medium

A. The 'values' parameter must be numeric, but 'Sales' contains strings.

B. The 'aggfunc' parameter must be a string, not a function.

C. The 'margins' parameter cannot be used with 'aggfunc'.

D. The DataFrame 'df' is missing the 'Category' column.

Solution

1. Step 1: Understand the interaction of margins and aggfunc

   Using margins=True with aggfunc other than 'count' or 'sum' can cause errors because totals for means are ambiguous.

2. Step 2: Check other options for errors

   Sales is numeric, so A is wrong; aggfunc='mean' is a valid string, so B is wrong; if 'Category' was missing, it would raise KeyError, not TypeError, so D is wrong.

3. Final Answer:

   The 'margins' parameter cannot be used with 'aggfunc'. -> Option C

4. Quick Check:

   margins with aggfunc='mean' causes error [OK]

Hint: Avoid margins=True with aggfunc='mean' to prevent errors [OK]

Common Mistakes:

• Assuming margins always works with aggfunc
• Confusing TypeError with KeyError
• Thinking 'values' must be strings

5. You have a DataFrame with columns Region, Product, and Profit. You want to create a cross-tabulation that shows the average profit per product in each region, including overall averages per region and product. Which code snippet correctly achieves this?

hard

A. pd.crosstab(df['Region'], df['Product'], values=df['Profit'], aggfunc='mean', margins=True)

B. pd.crosstab(df['Region'], df['Product'], values=df['Profit'], aggfunc='sum', margins=True)

C. pd.crosstab(df['Region'], df['Product'], values=df['Profit'], aggfunc='mean')

D. pd.crosstab(df['Region'], df['Product'], aggfunc='mean', margins=True)

Solution

1. Step 1: Identify the need for average profit with totals

   We want average profit per product per region, plus overall averages (totals), so aggfunc='mean' and margins=True are needed.

2. Step 2: Check the use of values parameter

   To aggregate profit, values=df['Profit'] must be specified; otherwise, crosstab counts occurrences.

3. Step 3: Evaluate options

   pd.crosstab(df['Region'], df['Product'], values=df['Profit'], aggfunc='mean', margins=True) uses mean, values, and margins correctly. pd.crosstab(df['Region'], df['Product'], values=df['Profit'], aggfunc='sum', margins=True) uses sum instead of mean. pd.crosstab(df['Region'], df['Product'], values=df['Profit'], aggfunc='mean') lacks margins. pd.crosstab(df['Region'], df['Product'], aggfunc='mean', margins=True) misses values parameter.

4. Final Answer:

   pd.crosstab(df['Region'], df['Product'], values=df['Profit'], aggfunc='mean', margins=True) -> Option A

5. Quick Check:

   Mean profit with totals = pd.crosstab(df['Region'], df['Product'], values=df['Profit'], aggfunc='mean', margins=True) [OK]

Hint: Use values + aggfunc='mean' + margins=True for averages with totals [OK]

Common Mistakes:

• Omitting values parameter
• Using sum instead of mean for average
• Forgetting margins=True for totals