groupby() basics in Data Analysis Python - Time & Space Complexity

Time Complexity: groupby() basics

O(n)

Understanding Time Complexity

When we use groupby() in data analysis, we want to know how long it takes as our data grows.

We ask: How does the time to group data change when we have more rows?

Scenario Under Consideration

Analyze the time complexity of the following code snippet.

import pandas as pd

data = pd.DataFrame({
    'Category': ['A', 'B', 'A', 'B', 'C'],
    'Value': [10, 20, 30, 40, 50]
})

grouped = data.groupby('Category').sum()

This code groups rows by the 'Category' column and sums the 'Value' for each group.

Identify Repeating Operations

Identify the loops, recursion, array traversals that repeat.

• Primary operation: Scanning each row once to assign it to a group.
• How many times: Once for each row in the data.

How Execution Grows With Input

As the number of rows grows, the time to group grows roughly the same way.

Input Size (n)	Approx. Operations
10	About 10 operations
100	About 100 operations
1000	About 1000 operations

Pattern observation: The time grows in a straight line with the number of rows.

Final Time Complexity

Time Complexity: O(n)

This means the time to group data grows directly with the number of rows.

Common Mistake

[X] Wrong: "Grouping data takes the same time no matter how many rows there are."

[OK] Correct: More rows mean more work to check and assign each row to a group, so time grows with data size.

Interview Connect

Understanding how grouping scales helps you explain data processing steps clearly and shows you know how data size affects performance.

Self-Check

"What if we grouped by two columns instead of one? How would the time complexity change?"

Practice

1. What does the groupby() function do in pandas?

easy

A. It splits data into groups based on column values.

B. It sorts the data in ascending order.

C. It removes duplicate rows from the data.

D. It changes the data type of columns.

Solution

1. Step 1: Understand the purpose of groupby()

   The groupby() function is used to split data into groups based on the values in one or more columns.

2. Step 2: Compare with other options

   Sorting, removing duplicates, and changing data types are different operations and not the main function of groupby().

3. Final Answer:

   It splits data into groups based on column values. -> Option A

4. Quick Check:

   groupby() = split data by column [OK]

Hint: Remember: groupby splits data into groups by column values [OK]

Common Mistakes:

• Confusing groupby with sorting or filtering
• Thinking groupby changes data types
• Assuming groupby removes duplicates

2. Which of the following is the correct syntax to group a DataFrame df by the column 'Category'?

easy

A. df.group_by('Category')

B. df.groupby('Category')

C. df.groupby['Category']

D. df.groupby(Category)

Solution

1. Step 1: Recall correct method syntax

   The correct method name is groupby without underscore, called with parentheses and column name as a string.

2. Step 2: Check each option

   The correct syntax is df.groupby('Category'). Incorrect versions include df.group_by('Category') (wrong method name), df.groupby['Category'] (wrong brackets), and df.groupby(Category) (missing quotes).

3. Final Answer:

   df.groupby('Category') -> Option B

4. Quick Check:

   Correct syntax uses parentheses and quotes [OK]

Hint: Use df.groupby('column_name') with quotes and parentheses [OK]

Common Mistakes:

• Using underscore in groupby (group_by)
• Using square brackets instead of parentheses
• Not quoting the column name

3. Given the DataFrame df below, what is the output of df.groupby('Team')['Points'].sum()?

   Team  Points
0  A     10
1  B     15
2  A     5
3  B     10
4  C     7

medium

A. Team
A 20
B 25
C 7
Name: Points, dtype: int64

B. Team
A 10
B 15
C 7
Name: Points, dtype: int64

C. Team
A 15
B 25
C 7
Name: Points, dtype: int64

D. Team
A 5
B 10
C 7
Name: Points, dtype: int64

Solution

1. Step 1: Group data by 'Team'

   The groups are: A with points 10 and 5, B with 15 and 10, C with 7.

2. Step 2: Sum points in each group

   Sum for A = 10 + 5 = 15, for B = 15 + 10 = 25, for C = 7.

3. Final Answer:

   Team A 15 B 25 C 7 Name: Points, dtype: int64 -> Option C

4. Quick Check:

   Sum points per team = 15, 25, 7 [OK]

Hint: Add values in each group to get sum quickly [OK]

Common Mistakes:

• Not summing all values per group
• Mixing group labels and sums
• Using count instead of sum

4. What is wrong with this code snippet?

grouped = df.groupby('Category')
result = grouped.sum['Sales']
print(result)

medium

A. Using square brackets instead of parentheses after sum.

B. Missing quotes around 'Category' in groupby.

C. Calling sum() without parentheses.

D. Using print() incorrectly.

Solution

1. Step 1: Check method call syntax

   The sum function must be called with parentheses: sum(), not accessed with square brackets.

2. Step 2: Verify other parts

   The groupby column is correctly quoted, and print usage is correct.

3. Final Answer:

   Using square brackets instead of parentheses after sum. -> Option A

4. Quick Check:

   sum() needs parentheses, not brackets [OK]

Hint: Call aggregation functions with parentheses like sum() [OK]

Common Mistakes:

• Using brackets instead of parentheses for functions
• Forgetting parentheses after sum
• Misquoting column names

5. You have a DataFrame df with columns 'City', 'Category', and 'Sales'. How do you find the average sales for each city and category combination?

hard

A. df.groupby('City' & 'Category')['Sales'].mean()

B. df.groupby('City', 'Category')['Sales'].mean()

C. df.groupby('City' + 'Category')['Sales'].mean()

D. df.groupby(['City', 'Category'])['Sales'].mean()

Solution

1. Step 1: Group by multiple columns

   To group by both 'City' and 'Category', pass a list of column names: ['City', 'Category'].

2. Step 2: Calculate mean of 'Sales'

   After grouping, select 'Sales' column and apply mean() to get average sales per group.

3. Final Answer:

   df.groupby(['City', 'Category'])['Sales'].mean() -> Option D

4. Quick Check:

   Group by list of columns, then mean() [OK]

Hint: Group by list of columns: df.groupby(['col1','col2']) [OK]

Common Mistakes:

• Passing multiple columns as separate arguments
• Using bitwise AND (&) instead of list
• Using string concatenation (+ ) instead of list