GroupBy with custom functions in Pandas - Time & Space Complexity

Time Complexity: GroupBy with custom functions

O(n)

Understanding Time Complexity

We want to understand how the time needed changes when using groupby with custom functions in pandas.

How does the work grow as the data gets bigger?

Scenario Under Consideration

Analyze the time complexity of the following code snippet.

import pandas as pd

def custom_func(group):
    return group.sum() + group.mean()

df = pd.DataFrame({
    'key': ['A', 'B', 'A', 'B', 'C'],
    'value': [1, 2, 3, 4, 5]
})
result = df.groupby('key')['value'].apply(custom_func)

This code groups data by 'key' and applies a custom function to each group.

Identify Repeating Operations

Identify the loops, recursion, array traversals that repeat.

• Primary operation: Applying the custom function to each group.
• How many times: Once per group, and inside the function, operations run over the group size.

How Execution Grows With Input

As the total data size grows, the number of groups and group sizes affect the work.

Input Size (n)	Approx. Operations
10	About 10 operations over groups and their sizes
100	About 100 operations, split across groups
1000	About 1000 operations, split across groups

Pattern observation: The total work grows roughly in proportion to the total data size.

Final Time Complexity

Time Complexity: O(n)

This means the time grows roughly in direct proportion to the number of rows in the data.

Common Mistake

[X] Wrong: "Grouping always makes the operation slower by a square factor."

[OK] Correct: Grouping splits data, but the total work still adds up to about the size of the data, not its square.

Interview Connect

Understanding how groupby with custom functions scales helps you explain data processing choices clearly and confidently.

Self-Check

"What if the custom function itself has a loop over the entire original data instead of just the group? How would the time complexity change?"