Single and multiple column grouping in Data Analysis Python - Time & Space Complexity

Time Complexity: Single and multiple column grouping

O(n)

Understanding Time Complexity

When we group data by one or more columns, the computer organizes rows into sets. We want to know how the time to do this changes as the data grows.

How does grouping time grow when we add more rows or columns?

Scenario Under Consideration

Analyze the time complexity of the following code snippet.

import pandas as pd

data = pd.DataFrame({
    'City': ['NY', 'LA', 'NY', 'LA', 'NY'],
    'Year': [2020, 2020, 2021, 2021, 2020],
    'Sales': [100, 200, 150, 250, 300]
})

result = data.groupby(['City', 'Year']).sum()

This code groups sales data by city and year, then sums sales in each group.

Identify Repeating Operations

Identify the loops, recursion, array traversals that repeat.

• Primary operation: Scanning each row once to assign it to a group.
• How many times: Exactly once per row in the data.

How Execution Grows With Input

As the number of rows grows, the grouping step must look at each row once to decide its group.

Input Size (n)	Approx. Operations
10	About 10 checks
100	About 100 checks
1000	About 1000 checks

Pattern observation: The number of operations grows directly with the number of rows.

Final Time Complexity

Time Complexity: O(n)

This means the time to group grows in a straight line with the number of rows.

Common Mistake

[X] Wrong: "Grouping by more columns makes the time grow much faster, like squared or worse."

[OK] Correct: Grouping still looks at each row once. More columns affect memory and grouping keys, but the main time is scanning rows, which grows linearly.

Interview Connect

Understanding how grouping scales helps you explain data processing steps clearly. It shows you can think about how data size affects performance, a useful skill in real projects.

Self-Check

"What if we grouped the data multiple times in a loop? How would the time complexity change?"

Practice

1. What does the groupby() function do in data analysis with pandas?

easy

A. It groups data based on one or more columns to summarize it.

B. It sorts the data alphabetically.

C. It deletes duplicate rows from the data.

D. It changes the data type of columns.

Solution

1. Step 1: Understand the purpose of grouping

   Grouping means collecting rows that share the same value in one or more columns.

2. Step 2: Role of groupby() in pandas

   The groupby() function in pandas helps to split data into groups based on column values for aggregation or analysis.

3. Final Answer:

   It groups data based on one or more columns to summarize it. -> Option A

4. Quick Check:

   Grouping = Summarizing data by columns [OK]

Hint: Remember: groupby = group rows by column values [OK]

Common Mistakes:

• Thinking groupby sorts data
• Confusing groupby with removing duplicates
• Assuming groupby changes data types

2. Which of the following is the correct syntax to group a pandas DataFrame df by two columns 'A' and 'B'?

easy

A. df.groupby(['A', 'B'])

B. df.groupby('A', 'B')

C. df.group_by(['A', 'B'])

D. df.groupby(['A' & 'B'])

Solution

1. Step 1: Check pandas groupby syntax for multiple columns

   To group by multiple columns, pass a list of column names inside groupby: df.groupby(['A', 'B']).

2. Step 2: Identify incorrect syntax options

   df.groupby('A', 'B') misses list brackets, df.group_by(['A', 'B']) uses wrong method name, df.groupby(['A' & 'B']) uses invalid operator inside list.

3. Final Answer:

   df.groupby(['A', 'B']) -> Option A

4. Quick Check:

   Multiple columns = list inside groupby [OK]

Hint: Use a list of columns inside groupby for multiple columns [OK]

Common Mistakes:

• Passing columns as separate arguments instead of list
• Using wrong method name like group_by
• Using operators inside list instead of commas

3. Given the DataFrame df below:

   City  Sales  Quantity
0  NY    100    2
1  LA    200    3
2  NY    150    1
3  LA    100    4
4  NY    200    2

What is the output of df.groupby('City')['Sales'].sum()?

medium

A. City
LA 400
NY 350
Name: Sales, dtype: int64

B. City
LA 400
NY 450
Name: Sales, dtype: int64

C. City
LA 300
NY 350
Name: Sales, dtype: int64

D. City
LA 300
NY 450
Name: Sales, dtype: int64

Solution

1. Step 1: Group data by 'City' and sum 'Sales'

   Group rows by City: LA rows have Sales 200 and 100, sum = 300; NY rows have Sales 100, 150, 200, sum = 450.

2. Step 2: Write the resulting Series

   The result shows sums per city: LA 300, NY 450, with City as index and Sales as name.

3. Final Answer:

   City LA 300 NY 450 Name: Sales, dtype: int64 -> Option D

4. Quick Check:

   Sum Sales by City = LA:300, NY:450 [OK]

Hint: Add values per group to find sum quickly [OK]

Common Mistakes:

• Adding wrong rows for each city
• Mixing sum of Sales with Quantity
• Confusing index and values in output

4. What is wrong with this code snippet?

df.groupby('Category', 'Type').mean()

medium

A. Category and Type columns must be numeric.

B. mean() cannot be used after groupby.

C. groupby expects a list for multiple columns, not separate arguments.

D. groupby requires column names as integers.

Solution

1. Step 1: Check groupby syntax for multiple columns

   When grouping by multiple columns, pass a list: df.groupby(['Category', 'Type']). Passing separate arguments causes error.

2. Step 2: Verify other options

   mean() is valid after groupby; columns can be non-numeric but mean applies only to numeric; column names are strings, not integers.

3. Final Answer:

   groupby expects a list for multiple columns, not separate arguments. -> Option C

4. Quick Check:

   Multiple columns need list in groupby [OK]

Hint: Use list brackets for multiple columns in groupby [OK]

Common Mistakes:

• Passing multiple columns as separate arguments
• Thinking mean() is invalid after groupby
• Assuming column names must be integers

5. You have a sales DataFrame df with columns 'Region', 'Product', and 'Revenue'. You want to find the average revenue per region and product, but only for products with total revenue above 1000. Which code correctly achieves this?

hard

A. df.groupby(['Region', 'Product'])['Revenue'].mean()[df['Revenue'].sum() > 1000]

B. df.groupby(['Region', 'Product']).filter(lambda x: x['Revenue'].sum() > 1000).groupby(['Region', 'Product'])['Revenue'].mean()

C. df.groupby(['Region', 'Product'])['Revenue'].sum().mean() > 1000

D. df.groupby(['Region', 'Product'])['Revenue'].mean().filter(lambda x: x > 1000)

Solution

1. Step 1: Filter groups with total revenue above 1000

   Use filter() with a lambda to keep groups where sum of Revenue > 1000.

2. Step 2: Calculate mean revenue per filtered group

   After filtering, group again by Region and Product and calculate mean Revenue.

3. Step 3: Analyze other options

   df.groupby(['Region', 'Product'])['Revenue'].mean()[df['Revenue'].sum() > 1000] uses total Revenue sum instead of per-group sum, returning all means or none; df.groupby(['Region', 'Product'])['Revenue'].sum().mean() > 1000 computes mean of sums, not per group; df.groupby(['Region', 'Product'])['Revenue'].mean().filter(lambda x: x > 1000) filters means > 1000, not sums.

4. Final Answer:

   df.groupby(['Region', 'Product']).filter(lambda x: x['Revenue'].sum() > 1000).groupby(['Region', 'Product'])['Revenue'].mean() -> Option B

5. Quick Check:

   Filter groups by sum, then compute mean [OK]

Hint: Use filter() to keep groups before aggregation [OK]

Common Mistakes:

• Trying to filter after aggregation with mismatched indexes
• Confusing mean of sums with sum of means
• Filtering on mean instead of sum