Aggregation with agg() in Pandas - Time & Space Complexity

Time Complexity: Aggregation with agg()

O(n)

Understanding Time Complexity

We want to understand how the time needed to aggregate data grows as the data gets bigger.

How does using agg() on a DataFrame scale with more rows?

Scenario Under Consideration

Analyze the time complexity of the following code snippet.

import pandas as pd

data = pd.DataFrame({
    'group': ['A', 'B', 'A', 'B', 'A'],
    'value': [10, 20, 30, 40, 50]
})

result = data.groupby('group').agg({'value': ['sum', 'mean']})

This code groups data by the 'group' column and calculates the sum and mean of 'value' for each group.

Identify Repeating Operations

Identify the loops, recursion, array traversals that repeat.

• Primary operation: Iterating over each row to assign it to a group and then aggregating values per group.
• How many times: Once for each row in the DataFrame during grouping, then once per group for aggregation.

How Execution Grows With Input

As the number of rows grows, the time to group and aggregate grows roughly in proportion to the number of rows.

Input Size (n)	Approx. Operations
10	About 10 operations to assign groups + aggregation
100	About 100 operations to assign groups + aggregation
1000	About 1000 operations to assign groups + aggregation

Pattern observation: The operations grow roughly linearly as the data size increases.

Final Time Complexity

Time Complexity: O(n)

This means the time to run agg() grows roughly in direct proportion to the number of rows.

Common Mistake

[X] Wrong: "Aggregation with agg() takes constant time no matter how big the data is."

[OK] Correct: The function must look at each row to group and calculate, so more rows mean more work.

Interview Connect

Understanding how aggregation scales helps you explain data processing speed clearly and shows you know how data size affects performance.

Self-Check

"What if we added multiple columns to aggregate with agg()? How would the time complexity change?"

Practice

1.

What does the agg() function do in pandas?

easy

A. It sorts the DataFrame rows by index.

B. It applies one or more aggregation functions to DataFrame columns.

C. It filters rows based on a condition.

D. It merges two DataFrames together.

Solution

1. Step 1: Understand the purpose of agg()

   The agg() function is used to apply aggregation functions like sum, mean, count, etc., to DataFrame columns.

2. Step 2: Compare with other options

   Sorting, filtering, and merging are done by other pandas functions, not agg().

3. Final Answer:

   It applies one or more aggregation functions to DataFrame columns. -> Option B

4. Quick Check:

   agg() = aggregation [OK]

Hint: Remember agg() means aggregate functions on columns [OK]

Common Mistakes:

• Confusing agg() with sorting or filtering functions
• Thinking agg() modifies row order
• Assuming agg() merges DataFrames

2.

Which of the following is the correct syntax to apply agg() with multiple functions on a DataFrame df?

df.agg({'col1': ['sum', 'mean'], 'col2': ['min', 'max']})

easy

A. df.agg({'col1': ['sum', 'mean'], 'col2': ['min', 'max']})

B. df.aggregate(['sum', 'mean'], ['min', 'max'])

C. df.agg(['col1': 'sum', 'col2': 'min'])

D. df.agg({'sum', 'mean', 'min', 'max'})

Solution

1. Step 1: Review correct agg() syntax

   The correct syntax uses a dictionary mapping column names to lists of functions: {'col1': ['sum', 'mean'], 'col2': ['min', 'max']}.

2. Step 2: Check each option

   df.agg({'col1': ['sum', 'mean'], 'col2': ['min', 'max']}) matches the correct syntax. Options A, B, and D have syntax errors or wrong argument formats.

3. Final Answer:

   df.agg({'col1': ['sum', 'mean'], 'col2': ['min', 'max']}) -> Option A

4. Quick Check:

   Dictionary with column keys and function lists = df.agg({'col1': ['sum', 'mean'], 'col2': ['min', 'max']}) [OK]

Hint: Use dict with columns as keys and list of funcs as values [OK]

Common Mistakes:

• Passing functions as separate arguments instead of dict
• Using sets {} instead of dict for agg()
• Incorrect list or dict syntax inside agg()

3.

Given the DataFrame:

import pandas as pd
df = pd.DataFrame({'A': [1, 2, 3], 'B': [4, 5, 6]})
result = df.agg({'A': ['sum', 'max'], 'B': ['sum', 'max']})
print(result)

What is the output?

medium

A. A B
sum 6 15
max 3 6

B. A B
sum 6 4
max 3 4

C. A B
sum 6 5
max 3 6

D. A B
sum 6 5
max 3 4

Solution

1. Step 1: Calculate aggregation for column 'A'

   Sum of [1, 2, 3] is 6; max is 3.

2. Step 2: Calculate aggregation for column 'B'

   Sum of [4, 5, 6] is 15; max is 6.

3. Final Answer:

   A B sum 6 15 max 3 6 -> Option A

4. Quick Check:

   Sum A=6, Max A=3, Sum B=15, Max B=6 [OK]

Hint: Apply each function only to specified columns [OK]

Common Mistakes:

• Assuming max applies to column B
• Mixing up min and max values
• Confusing row and column labels in output

4.

What is wrong with this code snippet?

import pandas as pd
df = pd.DataFrame({'X': [10, 20], 'Y': [30, 40]})
result = df.agg({'X': 'sum', 'Y': ['mean', 'max']})
print(result)

medium

A. Error because 'agg()' requires all functions as strings.

B. Error because 'sum' should be in a list for column 'X'.

C. Error because 'agg()' cannot mix single and multiple functions.

D. No error; code runs and prints correct aggregation.

Solution

1. Step 1: Check if mixing single and multiple functions is allowed

   In pandas, you can pass a single function as a string or multiple as a list per column in agg().

2. Step 2: Verify syntax correctness

   Passing 'sum' as a string for 'X' and ['mean', 'max'] for 'Y' is valid syntax.

3. Final Answer:

   No error; code runs and prints correct aggregation. -> Option D

4. Quick Check:

   Single or list functions per column both allowed [OK]

Hint: Single function can be string; multiple must be list [OK]

Common Mistakes:

• Thinking all functions must be lists
• Assuming mixing single and list causes error
• Confusing function names with strings vs objects

5.

You have a DataFrame df with columns 'Product', 'Sales', and 'Quantity'. You want to group by 'Product' and get the total sales, average quantity, and maximum sales per product. Which agg() call achieves this?

hard

A. df.groupby('Product').agg({'Sales': ['mean'], 'Quantity': ['sum', 'max']})

B. df.groupby('Product').agg({'Sales': 'mean', 'Quantity': ['sum', 'max']})

C. df.groupby('Product').agg({'Sales': ['sum', 'max'], 'Quantity': 'mean'})

D. df.groupby('Product').agg({'Product': ['sum'], 'Sales': 'max', 'Quantity': 'mean'})

Solution

1. Step 1: Identify required aggregations per column

   We want total sales (sum), average quantity (mean), and maximum sales (max) grouped by product.

2. Step 2: Match functions to columns

   For 'Sales', apply 'sum' and 'max'; for 'Quantity', apply 'mean'. Grouping is by 'Product'.

3. Step 3: Check options

   df.groupby('Product').agg({'Sales': ['sum', 'max'], 'Quantity': 'mean'}) matches the requirements exactly. Others have wrong functions or columns.

4. Final Answer:

   df.groupby('Product').agg({'Sales': ['sum', 'max'], 'Quantity': 'mean'}) -> Option C

5. Quick Check:

   Group by Product, Sales sum & max, Quantity mean = df.groupby('Product').agg({'Sales': ['sum', 'max'], 'Quantity': 'mean'}) [OK]

Hint: Map each column to needed functions in a dict inside agg() [OK]

Common Mistakes:

• Applying wrong functions to columns
• Including grouping column in agg()
• Mixing aggregation functions incorrectly