Aggregation functions (sum, mean, std) in Data Analysis Python - Time & Space Complexity

Time Complexity: Aggregation functions (sum, mean, std)

O(n)

Understanding Time Complexity

When we use aggregation functions like sum, mean, or standard deviation, we want to know how long they take as our data grows.

We ask: How does the time to calculate these values change when we have more data?

Scenario Under Consideration

Analyze the time complexity of the following code snippet.

import pandas as pd

data = pd.Series([1, 2, 3, 4, 5])

sum_value = data.sum()
mean_value = data.mean()
std_value = data.std()

This code calculates the sum, mean, and standard deviation of a list of numbers using pandas.

Identify Repeating Operations

Identify the loops, recursion, array traversals that repeat.

• Primary operation: Each aggregation function loops through all data points once.
• How many times: Each function visits every number in the list one time.

How Execution Grows With Input

As the number of data points grows, the time to calculate sum, mean, or std grows in a straight line.

Input Size (n)	Approx. Operations
10	About 10 steps
100	About 100 steps
1000	About 1000 steps

Pattern observation: Doubling the data roughly doubles the work needed.

Final Time Complexity

Time Complexity: O(n)

This means the time to compute these functions grows linearly with the number of data points.

Common Mistake

[X] Wrong: "Calculating mean or standard deviation is faster than sum because they are more complex."

[OK] Correct: All these functions need to look at every number at least once, so they take similar time that grows with data size.

Interview Connect

Understanding how aggregation functions scale helps you explain data processing speed clearly and confidently in real projects or interviews.

Self-Check

"What if we calculate the sum and mean together in one pass instead of separately? How would the time complexity change?"

Practice

1. What does the mean() function calculate in a dataset?

easy

A. The most frequent value in the data

B. The total sum of all data points

C. The difference between the largest and smallest values

D. The average value of the data

Solution

1. Step 1: Understand the purpose of mean()

   The mean() function calculates the average by adding all values and dividing by the count.

2. Step 2: Compare with other options

   Sum is total, range is difference, mode is most frequent. Only mean is average.

3. Final Answer:

   The average value of the data -> Option D

4. Quick Check:

   mean = average [OK]

Hint: Mean is average, sum is total, std is spread [OK]

Common Mistakes:

• Confusing mean with sum
• Thinking mean finds the most frequent value
• Mixing mean with range or mode

2. Which of the following is the correct way to calculate the standard deviation of a pandas DataFrame column named data?

easy

A. df.std(data)

B. std(df['data'])

C. df.data.std()

D. df['data'].standard_deviation()

Solution

1. Step 1: Recall pandas syntax for std()

   In pandas, to get std of a column, use df.column.std() or df['column'].std().

2. Step 2: Check each option

   df.data.std() uses correct dot notation. df.std(data) is wrong syntax. std(df['data']) is not a pandas function call. df['data'].standard_deviation() uses a non-existent method.

3. Final Answer:

   df.data.std() -> Option C

4. Quick Check:

   Use dot notation with std() [OK]

Hint: Use df.column.std() to get standard deviation [OK]

Common Mistakes:

• Calling std() as a standalone function
• Using wrong method names
• Passing column name as argument to std()

3. Given the following code:

import pandas as pd
df = pd.DataFrame({'values': [10, 20, 30, 40, 50]})
result = df['values'].sum()

What is the value of result?

medium

A. 150

B. 30

C. 25

D. 10

Solution

1. Step 1: Identify the data values

   The column 'values' contains [10, 20, 30, 40, 50].

2. Step 2: Calculate the sum

   Sum = 10 + 20 + 30 + 40 + 50 = 150.

3. Final Answer:

   150 -> Option A

4. Quick Check:

   Sum of values = 150 [OK]

Hint: Add all numbers to get sum [OK]

Common Mistakes:

• Calculating mean instead of sum
• Adding only first two numbers
• Confusing sum with count

4. The following code is intended to calculate the mean of a list but causes an error:

import pandas as pd
numbers = [5, 10, 15]
mean_value = numbers.mean()

What is the error and how to fix it?

medium

A. Error: 'list' object has no attribute 'mean'. Fix by converting list to pandas Series first.

B. Error: mean() requires two arguments. Fix by adding an argument.

C. Error: mean() is not imported. Fix by importing mean from pandas.

D. No error, code runs correctly.

Solution

1. Step 1: Identify the error cause

   Python lists do not have a mean() method, so calling numbers.mean() causes an AttributeError.

2. Step 2: Fix by converting list to pandas Series

   Convert list to pd.Series(numbers) then call .mean() to get the average.

3. Final Answer:

   Error: 'list' object has no attribute 'mean'. Fix by converting list to pandas Series first. -> Option A

4. Quick Check:

   Lists lack mean(), use pd.Series [OK]

Hint: Convert list to Series before calling mean() [OK]

Common Mistakes:

• Calling mean() directly on list
• Trying to import mean() separately
• Assuming mean() needs arguments

5. You have a DataFrame df with a column scores containing exam scores. You want to find the average score but only for scores above 50. Which code correctly calculates this?

hard

A. df['scores'].mean(df['scores'] > 50)

B. df[df['scores'] > 50]['scores'].mean()

C. df['scores'].sum() / 50

D. df['scores'].std()[df['scores'] > 50]

Solution

1. Step 1: Filter scores greater than 50

   Use df[df['scores'] > 50] to select rows where scores are above 50.

2. Step 2: Calculate mean of filtered scores

   Access the 'scores' column of filtered data and call .mean() to get average.

3. Final Answer:

   df[df['scores'] > 50]['scores'].mean() -> Option B

4. Quick Check:

   Filter then mean = correct approach [OK]

Hint: Filter first, then apply mean() [OK]

Common Mistakes:

• Passing condition inside mean()
• Dividing sum by 50 instead of count
• Using std() instead of mean()