Grouping by multiple columns in Pandas - Time & Space Complexity

Time Complexity: Grouping by multiple columns

O(n)

Understanding Time Complexity

When we group data by multiple columns, we want to see how the time to do this grows as the data gets bigger.

We ask: How does the work increase when we add more rows or more groups?

Scenario Under Consideration

Analyze the time complexity of the following code snippet.

import pandas as pd

data = pd.DataFrame({
    'City': ['NY', 'LA', 'NY', 'LA', 'NY'],
    'Year': [2020, 2020, 2021, 2021, 2020],
    'Sales': [100, 200, 150, 250, 300]
})

grouped = data.groupby(['City', 'Year']).sum()

This code groups sales data by both city and year, then sums the sales in each group.

Identify Repeating Operations

Identify the loops, recursion, array traversals that repeat.

• Primary operation: Scanning each row to find its group based on city and year.
• How many times: Once for each row in the data.

How Execution Grows With Input

As the number of rows grows, the time to group and sum grows roughly in direct proportion.

Input Size (n)	Approx. Operations
10	About 10 operations to assign groups and sum
100	About 100 operations
1000	About 1000 operations

Pattern observation: Doubling the rows roughly doubles the work needed.

Final Time Complexity

Time Complexity: O(n)

This means the time to group by multiple columns grows linearly with the number of rows.

Common Mistake

[X] Wrong: "Grouping by more columns makes the operation take much longer than just one column, like multiplying time by the number of columns."

[OK] Correct: Actually, grouping time depends mostly on the number of rows, not the number of grouping columns. Adding columns changes how groups are identified but does not multiply the work by the number of columns.

Interview Connect

Understanding how grouping scales helps you explain data processing steps clearly and shows you can think about efficiency in real tasks.

Self-Check

"What if we grouped by only one column instead of two? How would the time complexity change?"

Practice

1. What does grouping by multiple columns in pandas allow you to do?

easy

A. Filter rows based on a single condition

B. Sort data alphabetically by one column

C. Analyze data by several categories at the same time

D. Change the data type of multiple columns

Solution

1. Step 1: Understand grouping concept

   Grouping by multiple columns means splitting data into groups based on combinations of those columns.

2. Step 2: Identify the purpose of grouping

   This helps analyze data by several categories at once, like sales by region and product.

3. Final Answer:

   Analyze data by several categories at the same time -> Option C

4. Quick Check:

   Grouping multiple columns = multi-category analysis [OK]

Hint: Grouping by list of columns splits data by combined categories [OK]

Common Mistakes:

• Thinking grouping changes data order
• Confusing grouping with filtering
• Assuming grouping changes data types

2. Which of the following is the correct syntax to group a DataFrame df by columns 'A' and 'B'?

easy

A. df.groupby(['A', 'B'])

B. df.groupby('A', 'B')

C. df.group_by(['A', 'B'])

D. df.groupby(['A' & 'B'])

Solution

1. Step 1: Recall pandas groupby syntax

   To group by multiple columns, pass a list of column names inside groupby().

2. Step 2: Check each option

   df.groupby(['A', 'B']) uses a list ['A', 'B'], which is correct syntax. Others have syntax errors or wrong method names.

3. Final Answer:

   df.groupby(['A', 'B']) -> Option A

4. Quick Check:

   List of columns inside groupby() = correct syntax [OK]

Hint: Use a list of column names inside groupby() [OK]

Common Mistakes:

• Passing columns as separate arguments instead of a list
• Using wrong method name like group_by
• Using bitwise operators inside list

3. Given the DataFrame df below:

   City  Product  Sales
0  NY    A        10
1  NY    B        20
2  LA    A        15
3  LA    B        25
4  NY    A        5

What is the output of df.groupby(['City', 'Product']).sum()?

medium

A. A DataFrame with total sales for each City and Product combination

B. A DataFrame with average sales for each City

C. A Series with sales summed only by City

D. An error because sum() cannot be used after groupby

Solution

1. Step 1: Group by City and Product

   The data is grouped by both City and Product columns, creating groups like (NY, A), (NY, B), (LA, A), (LA, B).

2. Step 2: Apply sum() aggregation

   Sum sales within each group: (NY, A) = 10+5=15, (NY, B) = 20, (LA, A) = 15, (LA, B) = 25.

3. Final Answer:

   A DataFrame with total sales for each City and Product combination -> Option A

4. Quick Check:

   Group by multiple columns + sum() = grouped sums [OK]

Hint: Sum after groupby sums values per group combination [OK]

Common Mistakes:

• Expecting average instead of sum
• Grouping by only one column
• Thinking sum() causes error

4. What is wrong with this code snippet?

df.groupby('City', 'Product').mean()

medium

A. groupby() requires column names as integers

B. mean() cannot be used after groupby

C. The columns 'City' and 'Product' do not exist

D. groupby() expects a single argument, so multiple columns must be passed as a list

Solution

1. Step 1: Check groupby argument format

   groupby() accepts one argument for columns; multiple columns must be inside a list.

2. Step 2: Identify error cause

   Passing 'City', 'Product' as separate arguments causes a TypeError.

3. Final Answer:

   groupby() expects a single argument, so multiple columns must be passed as a list -> Option D

4. Quick Check:

   Multiple columns in groupby() = list argument [OK]

Hint: Pass multiple columns as a list inside groupby() [OK]

Common Mistakes:

• Passing multiple columns as separate arguments
• Thinking mean() is invalid after groupby
• Assuming column names must be integers

5. You have a sales DataFrame df with columns Region, Category, and Revenue. You want to find the average revenue per region and category, but only for categories where the total revenue exceeds 1000. Which code correctly does this?

hard

A. df.groupby(['Region', 'Category']).Revenue.mean()[df.groupby(['Region', 'Category']).Revenue.mean() > 1000]

B. df.groupby(['Region', 'Category']).filter(lambda x: x['Revenue'].sum() > 1000).groupby(['Region', 'Category']).Revenue.mean()

C. df.groupby(['Region', 'Category']).Revenue.mean().filter(lambda x: x > 1000)

D. df.groupby(['Region', 'Category']).Revenue.sum().mean() > 1000

Solution

1. Step 1: Filter groups by total revenue

   Use groupby with filter and a lambda to keep only groups where sum of Revenue > 1000.

2. Step 2: Calculate average revenue per filtered group

   After filtering, group again by Region and Category and calculate mean Revenue.

3. Final Answer:

   df.groupby(['Region', 'Category']).filter(lambda x: x['Revenue'].sum() > 1000).groupby(['Region', 'Category']).Revenue.mean() -> Option B

4. Quick Check:

   Filter groups by sum, then compute mean [OK]

Hint: Use filter() on groupby to keep groups before aggregation [OK]

Common Mistakes:

• Trying to filter after mean instead of before
• Using boolean indexing on Series with mismatched index
• Calculating mean of sums instead of per group