describe() for statistics in Data Analysis Python - Time & Space Complexity

Time Complexity: describe() for statistics

O(n)

Understanding Time Complexity

We want to understand how the time to run describe() changes as the data size grows.

How does the work inside describe() scale with more data?

Scenario Under Consideration

Analyze the time complexity of the following code snippet.

import pandas as pd

data = pd.Series([1, 2, 3, 4, 5])
summary = data.describe()
print(summary)

This code calculates basic statistics like count, mean, min, max, and quartiles for a data series.

Identify Repeating Operations

Identify the loops, recursion, array traversals that repeat.

• Primary operation: The function scans through all data points to compute statistics.
• How many times: Each statistic requires at least one pass over the data, but some can be combined.

How Execution Grows With Input

As the number of data points grows, the time to compute statistics grows roughly in a straight line.

Input Size (n)	Approx. Operations
10	About 10 operations per statistic
100	About 100 operations per statistic
1000	About 1000 operations per statistic

Pattern observation: Doubling the data roughly doubles the work needed.

Final Time Complexity

Time Complexity: O(n)

This means the time to run describe() grows linearly with the number of data points.

Common Mistake

[X] Wrong: "describe() runs instantly no matter how big the data is."

[OK] Correct: The function must look at every data point to calculate statistics, so more data means more work and more time.

Interview Connect

Understanding how basic statistics scale helps you explain data processing speed clearly and confidently in real projects.

Self-Check

"What if describe() was called on a DataFrame with many columns instead of a single Series? How would the time complexity change?"

Practice

1. What does the describe() function in pandas primarily provide for a DataFrame?

easy

A. A sorted version of the DataFrame

B. A plot of the data distribution

C. A list of all column names

D. A summary of main statistics like mean, median, and count

Solution

1. Step 1: Understand the purpose of describe()

   The describe() function summarizes key statistics such as count, mean, std, min, max, and quartiles for numeric data.

2. Step 2: Compare with other options

   Options B, C, and D describe different functions or outputs unrelated to describe().

3. Final Answer:

   A summary of main statistics like mean, median, and count -> Option D

4. Quick Check:

   describe() = summary statistics [OK]

Hint: Remember: describe() = quick stats summary [OK]

Common Mistakes:

• Thinking describe() creates plots
• Confusing describe() with listing columns
• Assuming describe() sorts data

2. Which of the following is the correct way to call describe() on a pandas DataFrame named df?

easy

A. df.describe()

B. describe(df)

C. df.describe

D. df.describe[]

Solution

1. Step 1: Recall pandas method syntax

   Methods are called with parentheses after the method name, so df.describe() is correct.

2. Step 2: Check other options for syntax errors

   df.describe[] uses invalid square brackets; describe(df) incorrectly calls it as a standalone function; df.describe misses parentheses and prints the method object.

3. Final Answer:

   df.describe() -> Option A

4. Quick Check:

   Method call needs () = df.describe() [OK]

Hint: Call methods with parentheses: df.describe() [OK]

Common Mistakes:

• Forgetting parentheses when calling methods
• Using square brackets instead of parentheses
• Trying to call describe as a standalone function

3. Given the DataFrame df below:

import pandas as pd
df = pd.DataFrame({'A': [1, 2, 3, 4, 5], 'B': [10, 20, 30, 40, 50]})
print(df.describe())

What is the output for the mean of column A?

medium

A. 3.0

B. 2.5

C. 15.0

D. 10.0

Solution

1. Step 1: Calculate mean of column A

   Column A has values [1, 2, 3, 4, 5]. Mean = (1+2+3+4+5)/5 = 15/5 = 3.0.

2. Step 2: Verify other options

   Options B, C, and D do not match the calculated mean.

3. Final Answer:

   3.0 -> Option A

4. Quick Check:

   Mean of A = 3.0 [OK]

Hint: Sum values and divide by count for mean [OK]

Common Mistakes:

• Mixing mean with median or mode
• Calculating sum instead of mean
• Using wrong column values

4. You run the code below but get unexpected output:

import pandas as pd
df = pd.DataFrame({'X': [5, 10, 15]})
print(df.describe)

What is the problem and how to fix it?

medium

A. DataFrame missing columns; add more columns

B. describe is not a pandas function; import missing

C. Missing parentheses after describe; fix with df.describe()

D. Syntax error in DataFrame creation

Solution

1. Step 1: Identify the problem

   The code references df.describe without parentheses, so it prints the method object, not the result.

2. Step 2: Fix the code

   Add parentheses to call the method: df.describe().

3. Final Answer:

   Missing parentheses after describe; fix with df.describe() -> Option C

4. Quick Check:

   Method call needs () = df.describe() [OK]

Hint: Always use () to call methods [OK]

Common Mistakes:

• Forgetting parentheses on method calls
• Assuming describe is not a method
• Thinking DataFrame needs more columns

5. You have a DataFrame df with numeric and categorical columns. You want describe() to include statistics for all columns, not just numeric ones. Which code will achieve this?

hard

A. df.describe(exclude='all')

B. df.describe(include='all')

C. df.describe(numeric_only=True)

D. df.describe(include='numeric')

Solution

1. Step 1: Understand describe() default behavior

   By default, describe() shows stats only for numeric columns.

2. Step 2: Use include='all' to get all columns

   Passing include='all' makes describe() summarize numeric and categorical columns.

3. Step 3: Check other options

   exclude='all' excludes all columns (invalid result); numeric_only=True limits to numeric only; include='numeric' matches default.

4. Final Answer:

   df.describe(include='all') -> Option B

5. Quick Check:

   include='all' = all columns summary [OK]

Hint: Use include='all' to describe all columns [OK]

Common Mistakes:

• Using numeric_only=True to get all columns
• Confusing include and exclude parameters
• Passing invalid values to include