groupby() basics in Pandas - Time & Space Complexity

Time Complexity: groupby() basics

O(n)

Understanding Time Complexity

We want to understand how the time needed to group data grows as the data gets bigger.

How does pandas groupby() handle larger datasets in terms of speed?

Scenario Under Consideration

Analyze the time complexity of the following code snippet.

import pandas as pd

data = pd.DataFrame({
    'Category': ['A', 'B', 'A', 'B', 'C', 'A'],
    'Value': [10, 20, 30, 40, 50, 60]
})

grouped = data.groupby('Category').sum()

This code groups rows by the 'Category' column and sums the 'Value' for each group.

Identify Repeating Operations

Identify the loops, recursion, array traversals that repeat.

• Primary operation: Scanning each row to assign it to a group.
• How many times: Once for each row in the data.

How Execution Grows With Input

As the number of rows grows, the time to group and sum grows roughly in direct proportion.

Input Size (n)	Approx. Operations
10	About 10 row checks and sums
100	About 100 row checks and sums
1000	About 1000 row checks and sums

Pattern observation: Doubling the rows roughly doubles the work done.

Final Time Complexity

Time Complexity: O(n)

This means the time grows linearly with the number of rows in the data.

Common Mistake

[X] Wrong: "Grouping data is instant no matter how big the data is."

[OK] Correct: Each row must be checked and assigned to a group, so more rows mean more work and more time.

Interview Connect

Knowing how grouping scales helps you explain your data processing choices clearly and confidently.

Self-Check

"What if we grouped by two columns instead of one? How would the time complexity change?"

Practice

1. What does the groupby() function do in pandas?

easy

A. It splits the data into groups based on column values.

B. It sorts the data in ascending order.

C. It removes duplicate rows from the data.

D. It changes the data type of columns.

Solution

1. Step 1: Understand the purpose of groupby()

   The groupby() function is used to split data into groups based on the values in one or more columns.

2. Step 2: Compare with other options

   Sorting, removing duplicates, and changing data types are different operations and not what groupby() does.

3. Final Answer:

   It splits the data into groups based on column values. -> Option A

4. Quick Check:

   groupby() = split data into groups [OK]

Hint: Remember: groupby splits data by column values [OK]

Common Mistakes:

• Confusing groupby with sorting
• Thinking groupby removes duplicates
• Assuming groupby changes data types

2. Which of the following is the correct syntax to group a DataFrame df by the column 'Category'?

easy

A. df.group_by('Category')

B. df.groupby['Category']

C. df.groupby('Category')

D. df.groupby(Category)

Solution

1. Step 1: Recall the correct method name and syntax

   The correct method is groupby() called with parentheses and the column name as a string argument.

2. Step 2: Check each option

   df.groupby('Category') uses df.groupby('Category') which is correct. df.group_by('Category') uses wrong method name group_by. df.groupby['Category'] uses square brackets instead of parentheses. df.groupby(Category) misses quotes around the column name.

3. Final Answer:

   df.groupby('Category') -> Option C

4. Quick Check:

   Correct syntax uses parentheses and string column name [OK]

Hint: Use df.groupby('column_name') with parentheses and quotes [OK]

Common Mistakes:

• Using wrong method name like group_by
• Using square brackets instead of parentheses
• Not quoting the column name string

3. Given the DataFrame:

import pandas as pd
data = {'Team': ['A', 'A', 'B', 'B', 'C'], 'Points': [10, 15, 10, 5, 20]}
df = pd.DataFrame(data)
result = df.groupby('Team').sum()

What is the output of result?

medium

A. Team Points
A 25
B 15
C 20

B. {'Team': ['A', 'B', 'C'], 'Points': [25, 15, 20]}

C. Team Points
A 10
B 10
C 20

D. Team Points
A 15
B 5
C 20

Solution

1. Step 1: Group data by 'Team'

   The groups are: A with points 10 and 15, B with points 10 and 5, C with points 20.

2. Step 2: Sum points in each group

   Sum for A = 10 + 15 = 25, B = 10 + 5 = 15, C = 20.

3. Final Answer:

   Team Points A 25 B 15 C 20 -> Option A

4. Quick Check:

   Sum points per team = 25, 15, 20 [OK]

Hint: Sum points after groupby to get totals per group [OK]

Common Mistakes:

• Not summing points, just taking first value
• Mixing up group sums
• Confusing output format

4. What is wrong with this code snippet?

df = pd.DataFrame({'City': ['X', 'Y', 'X'], 'Sales': [100, 200, 150]})
result = df.groupby.City.sum()

medium

A. DataFrame is missing import statement.

B. The column name 'City' is misspelled.

C. The sum() function cannot be used after groupby.

D. You cannot use dot notation after groupby; it should be parentheses.

Solution

1. Step 1: Check groupby syntax

   The correct way to group by a column is df.groupby('City') with parentheses and quotes.

2. Step 2: Identify the error in dot notation

   Using dot notation like df.groupby.City is invalid because groupby is a method, not an attribute.

3. Final Answer:

   You cannot use dot notation after groupby; it should be parentheses. -> Option D

4. Quick Check:

   Use parentheses with groupby, not dot notation [OK]

Hint: Always use parentheses with groupby, not dot notation [OK]

Common Mistakes:

• Using dot notation instead of parentheses
• Misspelling column names
• Thinking sum() is invalid after groupby

5. You have a sales DataFrame:

import pandas as pd
data = {'Store': ['A', 'A', 'B', 'B', 'B'], 'Product': ['X', 'Y', 'X', 'Y', 'X'], 'Sales': [100, 150, 200, 250, 300]}
df = pd.DataFrame(data)

How do you find the total sales for each store and product combination?

hard

A. df.groupby('Product').sum('Store')

B. df.groupby(['Store', 'Product']).sum()

C. df.groupby('Store').sum('Product')

D. df.groupby(['Store']).sum(['Product'])

Solution

1. Step 1: Understand grouping by multiple columns

   To group by both 'Store' and 'Product', pass a list of column names to groupby().

2. Step 2: Use sum() to aggregate sales

   After grouping, use sum() to add sales values for each group.

3. Final Answer:

   df.groupby(['Store', 'Product']).sum() -> Option B

4. Quick Check:

   Group by multiple columns using list, then sum [OK]

Hint: Group by list of columns for multi-level grouping [OK]

Common Mistakes:

• Passing column names to sum() instead of groupby()
• Grouping by single column only
• Using wrong syntax for groupby