Named aggregation in Pandas - Time & Space Complexity

Time Complexity: Named aggregation

O(n)

Understanding Time Complexity

We want to understand how the time needed to run named aggregation in pandas changes as the data grows.

Specifically, how does grouping and aggregating data scale with more rows?

Scenario Under Consideration

Analyze the time complexity of the following code snippet.

import pandas as pd

df = pd.DataFrame({
    'Category': ['A', 'B', 'A', 'B', 'C'],
    'Value1': [10, 20, 30, 40, 50],
    'Value2': [5, 10, 15, 20, 25]
})

result = df.groupby('Category').agg(
    total_value1=pd.NamedAgg(column='Value1', aggfunc='sum'),
    mean_value2=pd.NamedAgg(column='Value2', aggfunc='mean')
)

This code groups data by 'Category' and calculates the sum of 'Value1' and the mean of 'Value2' for each group.

Identify Repeating Operations

Identify the loops, recursion, array traversals that repeat.

• Primary operation: pandas loops internally over each row to assign it to a group and then performs aggregation on each group.
• How many times: Each of the n rows is processed once to find its group, then aggregation runs over each group's rows.

How Execution Grows With Input

As the number of rows grows, the time to group and aggregate grows roughly in direct proportion.

Input Size (n)	Approx. Operations
10	About 10 grouping and aggregation steps
100	About 100 grouping and aggregation steps
1000	About 1000 grouping and aggregation steps

Pattern observation: The work grows roughly linearly as the number of rows increases.

Final Time Complexity

Time Complexity: O(n)

This means the time needed grows roughly in direct proportion to the number of rows in the data.

Common Mistake

[X] Wrong: "Grouping and aggregation run in constant time no matter how big the data is."

[OK] Correct: The code must look at each row to assign it to a group and then combine values, so more rows mean more work.

Interview Connect

Understanding how grouping and aggregation scale helps you explain data processing efficiency clearly and confidently.

Self-Check

"What if we added multiple aggregation functions per column? How would the time complexity change?"

Practice

1. What is the main purpose of named aggregation in pandas?

easy

A. To assign custom names to aggregated summary columns

B. To rename columns before aggregation

C. To filter rows before grouping

D. To sort data after aggregation

Solution

1. Step 1: Understand aggregation in pandas

   Aggregation combines data by groups to produce summary statistics.

2. Step 2: Role of named aggregation

   Named aggregation lets you specify clear names for these summary columns, making results easier to read.

3. Final Answer:

   To assign custom names to aggregated summary columns -> Option A

4. Quick Check:

   Named aggregation = custom summary names [OK]

Hint: Named aggregation means naming summary columns clearly [OK]

Common Mistakes:

• Confusing named aggregation with renaming columns before aggregation
• Thinking it filters or sorts data
• Assuming it changes the grouping keys

2. Which of the following is the correct syntax for using named aggregation in groupby.agg()?

easy

A. df.groupby('A').agg({'new_col': 'B', 'sum'})

B. df.groupby('A').agg('new_col' = ('B', 'sum'))

C. df.groupby('A').agg(new_col=('B', 'sum'))

D. df.groupby('A').agg(('B', 'sum') = 'new_col')

Solution

1. Step 1: Recall named aggregation syntax

   Named aggregation uses keyword arguments with the format new_name=(column, function).

2. Step 2: Check each option

   df.groupby('A').agg(new_col=('B', 'sum')) matches this syntax exactly: new_col=('B', 'sum'). Others have invalid syntax or misplaced equals signs.

3. Final Answer:

   df.groupby('A').agg(new_col=('B', 'sum')) -> Option C

4. Quick Check:

   Named aggregation syntax = new_name=(col, func) [OK]

Hint: Use new_name=(column, function) inside agg() [OK]

Common Mistakes:

• Using equals sign inside dictionary instead of keyword argument
• Misplacing parentheses or quotes
• Trying to assign new_col outside agg()

3. Given the DataFrame:

import pandas as pd
df = pd.DataFrame({'Team': ['A', 'A', 'B', 'B'], 'Points': [10, 15, 10, 20], 'Assists': [5, 7, 8, 6]})
result = df.groupby('Team').agg(total_points=('Points', 'sum'), max_assists=('Assists', 'max'))
print(result)

What is the output?

medium

A. Team index with total_points as sum of Points and max_assists as max of Assists

B. DataFrame with columns Points and Assists unchanged

C. Error because named aggregation syntax is wrong

D. Empty DataFrame because groupby is missing

Solution

1. Step 1: Understand the groupby and aggregation

   Grouping by 'Team' groups rows by 'A' and 'B'. Then sum Points and max Assists per group.

2. Step 2: Check named aggregation results

   total_points sums Points per team: A=10+15=25, B=10+20=30; max_assists picks max Assists per team: A=max(5,7)=7, B=max(8,6)=8.

3. Final Answer:

   Team index with total_points as sum of Points and max_assists as max of Assists -> Option A

4. Quick Check:

   Named aggregation sums and maxes grouped data [OK]

Hint: Sum and max per group with named aggregation [OK]

Common Mistakes:

• Expecting original columns unchanged
• Confusing max with sum
• Thinking groupby needs reset_index() to work

4. What is wrong with this code snippet?

df.groupby('Category').agg(total=('Value', 'sum'), mean='Value')

medium

A. The sum function cannot be used with named aggregation

B. The groupby column 'Category' does not exist

C. The agg method requires a list, not keyword arguments

D. The mean aggregation is missing the function tuple syntax

Solution

1. Step 1: Review named aggregation syntax

   Each named aggregation must be a keyword argument with a tuple: new_name=(column, function).

2. Step 2: Identify the error

   The 'mean' aggregation is given as mean='Value' without specifying the aggregation function, which is incorrect.

3. Final Answer:

   The mean aggregation is missing the function tuple syntax -> Option D

4. Quick Check:

   Named aggregation needs (column, function) tuples [OK]

Hint: Each agg must be new_name=(col, func) [OK]

Common Mistakes:

• Passing column name as string without function
• Assuming agg accepts mixed syntax
• Using list instead of keyword arguments

5. You have a DataFrame with sales data:

import pandas as pd
df = pd.DataFrame({'Store': ['X', 'X', 'Y', 'Y', 'Z'], 'Sales': [100, 150, 200, 250, 300], 'Returns': [5, 3, 2, 4, 1]})

You want to create a summary DataFrame grouped by 'Store' with:

• Total sales named 'total_sales'
• Average returns named 'avg_returns'
• Count of sales entries named 'sales_count'

Which code correctly uses named aggregation to achieve this?

hard

A. df.groupby('Store').agg(sum('Sales') as total_sales, mean('Returns') as avg_returns, count('Sales') as sales_count)

B. df.groupby('Store').agg(total_sales=('Sales', 'sum'), avg_returns=('Returns', 'mean'), sales_count=('Sales', 'count'))

C. df.groupby('Store').agg(total_sales='sum', avg_returns='mean', sales_count='count')

D. df.groupby('Store').agg({'total_sales': ('Sales', 'sum'), 'avg_returns': ('Returns', 'mean'), 'sales_count': ('Sales', 'count')})

Solution

1. Step 1: Recall named aggregation syntax

   Use keyword arguments with new_name=(column, function) inside agg().

2. Step 2: Evaluate each option

   df.groupby('Store').agg(total_sales=('Sales', 'sum'), avg_returns=('Returns', 'mean'), sales_count=('Sales', 'count')) correctly uses named aggregation syntax. df.groupby('Store').agg({'total_sales': ('Sales', 'sum'), 'avg_returns': ('Returns', 'mean'), 'sales_count': ('Sales', 'count')}) uses a dictionary but this is not valid for named aggregation. df.groupby('Store').agg(total_sales='sum', avg_returns='mean', sales_count='count') uses strings without specifying columns. df.groupby('Store').agg(sum('Sales') as total_sales, mean('Returns') as avg_returns, count('Sales') as sales_count) uses invalid syntax.

3. Final Answer:

   df.groupby('Store').agg(total_sales=('Sales', 'sum'), avg_returns=('Returns', 'mean'), sales_count=('Sales', 'count')) -> Option B

4. Quick Check:

   Named aggregation = keyword args with (col, func) tuples [OK]

Hint: Use new_name=(col, func) for multiple summaries [OK]

Common Mistakes:

• Using dictionary instead of keyword arguments
• Passing only function names without columns
• Trying SQL-like syntax inside agg()