Multiple aggregation functions in Pandas - Time & Space Complexity

Time Complexity: Multiple aggregation functions

O(n)

Understanding Time Complexity

We want to understand how the time needed to run multiple aggregation functions on data grows as the data gets bigger.

How does adding more data affect the work pandas does when summarizing it?

Scenario Under Consideration

Analyze the time complexity of the following code snippet.

import pandas as pd

df = pd.DataFrame({
    'group': ['A', 'B', 'A', 'B', 'A', 'B'],
    'value': [10, 20, 30, 40, 50, 60]
})

result = df.groupby('group').agg({'value': ['sum', 'mean', 'max']})

This code groups data by a column and calculates sum, mean, and max for each group.

Identify Repeating Operations

• Primary operation: pandas loops internally over each group to compute each aggregation.
• How many times: For each group, it processes all rows once per aggregation function.

How Execution Grows With Input

As the number of rows grows, pandas must look at more data for each aggregation.

Input Size (n)	Approx. Operations
10	About 30 (10 rows x 3 functions)
100	About 300 (100 rows x 3 functions)
1000	About 3000 (1000 rows x 3 functions)

Pattern observation: The work grows roughly in direct proportion to the number of rows and the number of aggregation functions.

Final Time Complexity

Time Complexity: O(n)

This means the time to run these aggregations grows linearly with the number of rows in the data.

Common Mistake

[X] Wrong: "Adding more aggregation functions multiplies the time by the square of the data size."

[OK] Correct: Each aggregation looks at the data once, so time grows with data size times number of functions, not squared.

Interview Connect

Knowing how aggregation time grows helps you explain performance when working with grouped data, a common task in data analysis.

Self-Check

"What if we added more grouping columns? How would the time complexity change?"

Practice

1. What does the agg() function do when used after groupby() in pandas?

easy

A. It merges two DataFrames together.

B. It sorts the groups in ascending order.

C. It filters rows based on a condition.

D. It applies multiple aggregation functions to each group and returns their results.

Solution

1. Step 1: Understand groupby operation

   The groupby() function splits data into groups based on column values.

2. Step 2: Apply agg() after grouping

   The agg() function applies one or more aggregation functions like sum, mean, count to each group and returns the results.

3. Final Answer:

   It applies multiple aggregation functions to each group and returns their results. -> Option D

4. Quick Check:

   agg() summarizes groups [OK]

Hint: Remember: agg() summarizes groups with many functions [OK]

Common Mistakes:

• Thinking agg() sorts data
• Confusing agg() with filtering
• Assuming agg() merges DataFrames

2. Which of the following is the correct syntax to apply both sum and mean aggregations on a grouped DataFrame df grouped by column 'Category'?

easy

A. df.groupby('Category').agg(['sum', 'mean'])

B. df.agg(['sum', 'mean']).groupby('Category')

C. df.groupby('Category').agg('sum', 'mean')

D. df.groupby('Category').aggregate(sum, mean)

Solution

1. Step 1: Identify correct method chaining

   Grouping must happen first with groupby('Category'), then aggregation with agg().

2. Step 2: Use list of functions inside agg()

   The correct syntax uses a list of function names: agg(['sum', 'mean']).

3. Final Answer:

   df.groupby('Category').agg(['sum', 'mean']) -> Option A

4. Quick Check:

   Group first, then agg with list [OK]

Hint: Use list inside agg() after groupby() [OK]

Common Mistakes:

• Calling agg() before groupby()
• Passing multiple args instead of list to agg()
• Using aggregate() without list syntax

3. Given the DataFrame:

import pandas as pd
data = {'Team': ['A', 'A', 'B', 'B'], 'Points': [10, 15, 10, 20], 'Assists': [5, 7, 8, 6]}
df = pd.DataFrame(data)
result = df.groupby('Team').agg({'Points': ['sum', 'mean'], 'Assists': 'max'})
print(result)

What is the output?

medium

A. A DataFrame with sum and mean of Points and max of Assists for each Team.

B. A DataFrame with sum and mean of Assists and max of Points for each Team.

C. An error because agg() cannot mix list and string in dict.

D. A Series with aggregated values for all columns combined.

Solution

1. Step 1: Group by 'Team'

   The data is grouped into two groups: Team A and Team B.

2. Step 2: Apply multiple aggregations

   For 'Points', sum and mean are calculated; for 'Assists', max is calculated for each group.

3. Final Answer:

   A DataFrame with sum and mean of Points and max of Assists for each Team. -> Option A

4. Quick Check:

   agg() supports dict with list and string [OK]

Hint: Use dict with list/string in agg() for different columns [OK]

Common Mistakes:

• Expecting error mixing list and string in agg dict
• Confusing which function applies to which column
• Thinking output is a Series, not DataFrame

4. What is wrong with this code snippet?

df.groupby('Category').agg({'Sales': 'sum', 'Profit': ['mean', 'max']})

It raises an error. Why?

medium

A. agg() does not accept multiple functions for different columns.

B. The column names 'Sales' and 'Profit' must be passed as a list, not dict.

C. The code is correct and runs without error.

D. The dictionary mixes single string and list of functions for different columns, which is invalid.

Solution

1. Step 1: Check agg() dictionary syntax

   agg() accepts a dictionary where values can be a string or list of functions per column.

2. Step 2: Verify code correctness

   The code correctly applies 'sum' to 'Sales' and ['mean', 'max'] to 'Profit' without error.

3. Final Answer:

   The code is correct and runs without error. -> Option C

4. Quick Check:

   agg() accepts mixed string and list in dict [OK]

Hint: agg() allows string or list per column in dict [OK]

Common Mistakes:

• Thinking mixing string and list causes error
• Expecting all functions must be in a list
• Confusing agg() with apply() syntax

5. You have a DataFrame df with columns 'Store', 'Sales', and 'Expenses'. You want to group by 'Store' and get the total sales, average expenses, and count of sales entries per store in one step. Which code correctly does this?

hard

A. df.groupby('Store').agg({'Sales': 'sum', 'Expenses': ['mean', 'count']})

B. df.groupby('Store').agg({'Sales': ['sum', 'count'], 'Expenses': 'mean'})

C. df.groupby('Store').agg({'Sales': 'mean', 'Expenses': 'sum', 'Store': 'count'})

D. df.groupby('Store').agg(['sum', 'mean', 'count'])

Solution

1. Step 1: Identify required aggregations per column

   We want total sales (sum), average expenses (mean), and count of sales entries (count).

2. Step 2: Map functions correctly in agg()

   Use a dict: 'Sales' gets ['sum', 'count'], 'Expenses' gets 'mean'. This matches df.groupby('Store').agg({'Sales': ['sum', 'count'], 'Expenses': 'mean'}).

3. Final Answer:

   df.groupby('Store').agg({'Sales': ['sum', 'count'], 'Expenses': 'mean'}) -> Option B

4. Quick Check:

   Dict with list and string per column [OK]

Hint: Use dict with list for multiple funcs per column [OK]

Common Mistakes:

• Putting count under wrong column
• Using agg() without dict for columns
• Applying all functions to all columns