Aggregation functions (sum, mean, count) in Data Analysis Python - Time & Space Complexity

Time Complexity: Aggregation functions (sum, mean, count)

O(n)

Understanding Time Complexity

When we use aggregation functions like sum, mean, or count on data, we want to know how long it takes as the data grows.

We ask: How does the time to calculate these values change when we have more data?

Scenario Under Consideration

Analyze the time complexity of the following code snippet.

import pandas as pd

data = pd.Series([1, 2, 3, 4, 5])

result_sum = data.sum()
result_mean = data.mean()
result_count = data.count()

This code calculates the sum, mean, and count of numbers in a list using pandas.

Identify Repeating Operations

Identify the loops, recursion, array traversals that repeat.

• Primary operation: Each aggregation function goes through all data items once.
• How many times: Each function scans the entire list of numbers one time.

How Execution Grows With Input

As the number of data points grows, the time to calculate sum, mean, or count grows in a straight line.

Input Size (n)	Approx. Operations
10	About 10 steps
100	About 100 steps
1000	About 1000 steps

Pattern observation: Doubling the data roughly doubles the work needed.

Final Time Complexity

Time Complexity: O(n)

This means the time to compute sum, mean, or count grows directly with the number of data points.

Common Mistake

[X] Wrong: "Aggregation functions like sum or mean take the same time no matter how big the data is."

[OK] Correct: These functions must look at each item once, so more data means more work and more time.

Interview Connect

Understanding how aggregation time grows helps you explain data processing speed clearly and confidently in real situations.

Self-Check

"What if we used a pre-calculated running total instead of summing all data each time? How would the time complexity change?"

Practice

1. What does the sum() function do when applied to a column of numbers in a DataFrame?

easy

A. Counts how many numbers are in the column

B. Finds the middle value of the numbers

C. Adds all the numbers together to get a total

D. Sorts the numbers from smallest to largest

Solution

1. Step 1: Understand the purpose of sum()

   The sum() function adds all values in a column to get their total.

2. Step 2: Compare with other functions

   mean() finds average, count() counts entries, so they do different things.

3. Final Answer:

   Adds all the numbers together to get a total -> Option C

4. Quick Check:

   sum() = total of values [OK]

Hint: Sum adds values, mean averages, count counts entries [OK]

Common Mistakes:

• Confusing sum() with mean()
• Thinking sum() counts entries
• Assuming sum() sorts data

2. Which of the following is the correct way to calculate the average of a DataFrame column named sales using pandas?

easy

A. df.sales.sum() / df.sales.count()

B. df.mean('sales')

C. mean(df.sales)

D. df['sales'].mean()

Solution

1. Step 1: Identify correct pandas syntax for mean()

   In pandas, to get the mean of a column, use df['column'].mean().

2. Step 2: Check other options

   df.mean('sales') is invalid syntax; mean(df.sales) is not pandas method; df.sales.sum() / df.sales.count() works but is longer and less direct.

3. Final Answer:

   df['sales'].mean() -> Option D

4. Quick Check:

   Use df['column'].mean() for average [OK]

Hint: Use df['col'].mean() to get average quickly [OK]

Common Mistakes:

• Using mean() as a standalone function
• Passing column name as argument to df.mean()
• Dividing sum by count manually instead of mean()

3. Given the DataFrame df below:

import pandas as pd
data = {'score': [10, 20, 30, 40, 50]}
df = pd.DataFrame(data)

What is the output of df['score'].count()?

medium

A. 5

B. 150

C. 30

D. Error

Solution

1. Step 1: Understand what count() does on a column

   count() counts the number of non-missing entries in the column. Here, all 5 values are present.

2. Step 2: Count the entries in 'score'

   There are 5 numbers: 10, 20, 30, 40, 50, so count() returns 5.

3. Final Answer:

   5 -> Option A

4. Quick Check:

   count() = number of entries [OK]

Hint: count() returns number of non-empty entries [OK]

Common Mistakes:

• Confusing count() with sum()
• Expecting count() to return total value
• Thinking count() returns average

4. Consider this code snippet:

import pandas as pd
data = {'values': [5, None, 15, 20]}
df = pd.DataFrame(data)
result = df['values'].sum()

What is the value of result?

medium

A. 40

B. None

C. Error

D. 20

Solution

1. Step 1: Understand how sum() handles None values

   In pandas, sum() ignores None (missing) values and sums only the numbers.

2. Step 2: Calculate sum ignoring None

   Values are 5, None, 15, 20. Sum = 5 + 15 + 20 = 40.

3. Final Answer:

   40 -> Option A

4. Quick Check:

   sum() skips None and adds numbers [OK]

Hint: sum() ignores None/null values automatically [OK]

Common Mistakes:

• Expecting sum() to return None if any missing
• Getting error due to None value
• Counting None as zero explicitly

5. You have a DataFrame df with a column sales containing daily sales numbers. You want to find the average sales but only for days where sales were above zero. Which code correctly calculates this?

hard

A. df['sales'].mean()

B. df[df['sales'] > 0]['sales'].mean()

C. df['sales'].sum() / df['sales'].count()

D. df['sales'].count() / df['sales'].sum()

Solution

1. Step 1: Filter rows where sales > 0

   Use df[df['sales'] > 0] to keep only positive sales days.

2. Step 2: Calculate mean of filtered sales

   Apply ['sales'].mean() on filtered data to get average of positive sales only.

3. Final Answer:

   df[df['sales'] > 0]['sales'].mean() -> Option B

4. Quick Check:

   Filter then mean() for conditional average [OK]

Hint: Filter data first, then apply mean() for conditional average [OK]

Common Mistakes:

• Using mean() without filtering
• Dividing sum by count without filtering
• Swapping sum and count in division