Grouping by multiple columns in Pandas

Introduction

Grouping by multiple columns helps you organize data into smaller parts based on several categories at once. This makes it easier to analyze patterns and summaries for each group.

You want to find the total sales for each product in each store.

You need to calculate the average score for students by both class and subject.

You want to count how many customers bought each product in different regions.

You want to see the maximum temperature recorded for each city by month.

Syntax

Pandas

df.groupby(['column1', 'column2', ...])['target_column'].aggregation_function()

You list the columns to group by inside a list.

After grouping, you choose which column to aggregate and what function to use, like sum(), mean(), or count().

Examples

Sum sales for each city and month combination.

Pandas

df.groupby(['City', 'Month'])['Sales'].sum()

Calculate average score for each class and subject.

Pandas

df.groupby(['Class', 'Subject'])['Score'].mean()

Count how many times each product was sold in each store.

Pandas

df.groupby(['Store', 'Product'])['Quantity'].count()

Sample Program

This code groups sales data by both store and product. Then it adds up the sales numbers for each group. The result shows total sales for each product in each store.

Pandas

import pandas as pd

data = {
    'Store': ['A', 'A', 'B', 'B', 'A', 'B'],
    'Product': ['Apple', 'Banana', 'Apple', 'Banana', 'Apple', 'Banana'],
    'Sales': [10, 15, 5, 20, 7, 8]
}

df = pd.DataFrame(data)

# Group by Store and Product, then sum the Sales
result = df.groupby(['Store', 'Product'])['Sales'].sum()

print(result)

Important Notes

Grouping by multiple columns creates a multi-level index in the result.

You can reset the index with reset_index() if you want a flat table.

Aggregation functions can be sum(), mean(), count(), max(), min(), and more.

Summary

Grouping by multiple columns helps analyze data by several categories at once.

Use a list of column names inside groupby() to group by more than one column.

Apply aggregation functions to get summaries like sums or averages for each group.

Practice

1. What does grouping by multiple columns in pandas allow you to do?

easy

A. Filter rows based on a single condition

B. Sort data alphabetically by one column

C. Analyze data by several categories at the same time

D. Change the data type of multiple columns

Solution

1. Step 1: Understand grouping concept

   Grouping by multiple columns means splitting data into groups based on combinations of those columns.

2. Step 2: Identify the purpose of grouping

   This helps analyze data by several categories at once, like sales by region and product.

3. Final Answer:

   Analyze data by several categories at the same time -> Option C

4. Quick Check:

   Grouping multiple columns = multi-category analysis [OK]

Hint: Grouping by list of columns splits data by combined categories [OK]

Common Mistakes:

• Thinking grouping changes data order
• Confusing grouping with filtering
• Assuming grouping changes data types

2. Which of the following is the correct syntax to group a DataFrame df by columns 'A' and 'B'?

easy

A. df.groupby(['A', 'B'])

B. df.groupby('A', 'B')

C. df.group_by(['A', 'B'])

D. df.groupby(['A' & 'B'])

Solution

1. Step 1: Recall pandas groupby syntax

   To group by multiple columns, pass a list of column names inside groupby().

2. Step 2: Check each option

   df.groupby(['A', 'B']) uses a list ['A', 'B'], which is correct syntax. Others have syntax errors or wrong method names.

3. Final Answer:

   df.groupby(['A', 'B']) -> Option A

4. Quick Check:

   List of columns inside groupby() = correct syntax [OK]

Hint: Use a list of column names inside groupby() [OK]

Common Mistakes:

• Passing columns as separate arguments instead of a list
• Using wrong method name like group_by
• Using bitwise operators inside list

3. Given the DataFrame df below:

   City  Product  Sales
0  NY    A        10
1  NY    B        20
2  LA    A        15
3  LA    B        25
4  NY    A        5

What is the output of df.groupby(['City', 'Product']).sum()?

medium

A. A DataFrame with total sales for each City and Product combination

B. A DataFrame with average sales for each City

C. A Series with sales summed only by City

D. An error because sum() cannot be used after groupby

Solution

1. Step 1: Group by City and Product

   The data is grouped by both City and Product columns, creating groups like (NY, A), (NY, B), (LA, A), (LA, B).

2. Step 2: Apply sum() aggregation

   Sum sales within each group: (NY, A) = 10+5=15, (NY, B) = 20, (LA, A) = 15, (LA, B) = 25.

3. Final Answer:

   A DataFrame with total sales for each City and Product combination -> Option A

4. Quick Check:

   Group by multiple columns + sum() = grouped sums [OK]

Hint: Sum after groupby sums values per group combination [OK]

Common Mistakes:

• Expecting average instead of sum
• Grouping by only one column
• Thinking sum() causes error

4. What is wrong with this code snippet?

df.groupby('City', 'Product').mean()

medium

A. groupby() requires column names as integers

B. mean() cannot be used after groupby

C. The columns 'City' and 'Product' do not exist

D. groupby() expects a single argument, so multiple columns must be passed as a list

Solution

1. Step 1: Check groupby argument format

   groupby() accepts one argument for columns; multiple columns must be inside a list.

2. Step 2: Identify error cause

   Passing 'City', 'Product' as separate arguments causes a TypeError.

3. Final Answer:

   groupby() expects a single argument, so multiple columns must be passed as a list -> Option D

4. Quick Check:

   Multiple columns in groupby() = list argument [OK]

Hint: Pass multiple columns as a list inside groupby() [OK]

Common Mistakes:

• Passing multiple columns as separate arguments
• Thinking mean() is invalid after groupby
• Assuming column names must be integers

5. You have a sales DataFrame df with columns Region, Category, and Revenue. You want to find the average revenue per region and category, but only for categories where the total revenue exceeds 1000. Which code correctly does this?

hard

A. df.groupby(['Region', 'Category']).Revenue.mean()[df.groupby(['Region', 'Category']).Revenue.mean() > 1000]

B. df.groupby(['Region', 'Category']).filter(lambda x: x['Revenue'].sum() > 1000).groupby(['Region', 'Category']).Revenue.mean()

C. df.groupby(['Region', 'Category']).Revenue.mean().filter(lambda x: x > 1000)

D. df.groupby(['Region', 'Category']).Revenue.sum().mean() > 1000

Solution

1. Step 1: Filter groups by total revenue

   Use groupby with filter and a lambda to keep only groups where sum of Revenue > 1000.

2. Step 2: Calculate average revenue per filtered group

   After filtering, group again by Region and Category and calculate mean Revenue.

3. Final Answer:

   df.groupby(['Region', 'Category']).filter(lambda x: x['Revenue'].sum() > 1000).groupby(['Region', 'Category']).Revenue.mean() -> Option B

4. Quick Check:

   Filter groups by sum, then compute mean [OK]

Hint: Use filter() on groupby to keep groups before aggregation [OK]

Common Mistakes:

• Trying to filter after mean instead of before
• Using boolean indexing on Series with mismatched index
• Calculating mean of sums instead of per group