First data analysis walkthrough in Data Analysis Python - Time & Space Complexity

Time Complexity: First data analysis walkthrough

O(n)

Understanding Time Complexity

When we analyze data, we often run code that looks at many rows. Understanding how the time to run this code grows as the data grows helps us plan better.

We want to know: How does the work change when we have more data?

Scenario Under Consideration

Analyze the time complexity of the following code snippet.

import pandas as pd

def analyze_data(df):
    total = 0
    for value in df['numbers']:
        total += value
    return total

# df is a DataFrame with a column 'numbers'

This code adds up all the numbers in one column of a data table.

Identify Repeating Operations

Identify the loops, recursion, array traversals that repeat.

• Primary operation: Looping through each number in the column.
• How many times: Once for every row in the data.

How Execution Grows With Input

As the number of rows grows, the time to add all numbers grows at the same pace.

Input Size (n)	Approx. Operations
10	10 additions
100	100 additions
1000	1000 additions

Pattern observation: Doubling the data doubles the work.

Final Time Complexity

Time Complexity: O(n)

This means the time to finish grows directly with the number of rows.

Common Mistake

[X] Wrong: "Adding numbers in a column takes the same time no matter how big the data is."

[OK] Correct: Each number must be looked at once, so more data means more work.

Interview Connect

Knowing how your code grows with data size shows you understand efficiency. This helps you write better data analysis code and explain your thinking clearly.

Self-Check

"What if we used a built-in function like sum() instead of a loop? How would the time complexity change?"

Practice

1. What is the main purpose of using head() on a DataFrame during the first data analysis walkthrough?

easy

A. To check the data types of each column

B. To get summary statistics of numerical columns

C. To quickly see the first few rows of the data

D. To remove missing values from the data

Solution

1. Step 1: Understand the function head()

   The head() function shows the first few rows of a DataFrame, usually the first 5 rows by default.

2. Step 2: Purpose in data analysis walkthrough

   Seeing the first few rows helps you quickly understand what kind of data you have and check if it loaded correctly.

3. Final Answer:

   To quickly see the first few rows of the data -> Option C

4. Quick Check:

   head() shows first rows [OK]

Hint: Remember: head() shows top rows fast [OK]

Common Mistakes:

• Confusing head() with describe()
• Thinking head() gives statistics
• Assuming head() cleans data

2. Which of the following is the correct syntax to get a summary of data types and non-null counts in a DataFrame named df?

easy

A. df.describe()

B. df.info()

C. df.head()

D. df.summary()

Solution

1. Step 1: Identify the function for data types and non-null counts

   The info() method on a DataFrame shows data types and counts of non-null values per column.

2. Step 2: Match syntax with DataFrame df

   Calling df.info() correctly applies this method to the DataFrame named df.

3. Final Answer:

   df.info() -> Option B

4. Quick Check:

   info() shows data types and counts [OK]

Hint: Use info() to check data types and missing data [OK]

Common Mistakes:

• Using describe() to check data types
• Trying summary() which does not exist
• Confusing head() with info()

3. Given the following code:

import pandas as pd
data = {'age': [25, 30, 22, 40, 28], 'salary': [50000, 60000, 45000, 80000, 52000]}
df = pd.DataFrame(data)
summary = df.describe()

What will be the output of summary.loc['mean', 'salary']?

medium

A. 52000.0

B. SyntaxError

C. 60000.0

D. 57400.0

Solution

1. Step 1: Calculate the mean of the 'salary' column

   Sum salaries: 50000 + 60000 + 45000 + 80000 + 52000 = 287000. Number of entries: 5. Mean = 287000 / 5 = 57400.0.

2. Step 2: Understand describe() output

   The describe() method returns a DataFrame with statistics including 'mean' as an index row. Accessing summary.loc['mean', 'salary'] gives the mean salary value.

3. Final Answer:

   57400.0 -> Option D

4. Quick Check:

   Mean salary = 57400.0 [OK]

Hint: Sum values and divide by count for mean [OK]

Common Mistakes:

• Adding incorrectly or missing one value
• Confusing mean with median or mode
• Using wrong index or column name

4. You run the following code but get an error:

import pandas as pd
data = {'name': ['Alice', 'Bob'], 'age': [25, 30]}
df = pd.DataFrame(data)
print(df.describe())

What is the most likely reason for the error or unexpected output?

medium

A. The describe() method cannot handle string columns by default

B. The DataFrame is missing a numeric column

C. The DataFrame is empty

D. The describe() method requires a parameter

Solution

1. Step 1: Check DataFrame columns and types

   The DataFrame has a string column 'name' and a numeric column 'age'.

2. Step 2: Understand describe() behavior

   By default, describe() summarizes only numeric columns. It ignores string columns unless specified.

3. Final Answer:

   The describe() method cannot handle string columns by default -> Option A

4. Quick Check:

   describe() skips strings by default [OK]

Hint: describe() defaults to numeric columns only [OK]

Common Mistakes:

• Expecting describe() to summarize strings automatically
• Thinking DataFrame is empty
• Assuming describe() needs parameters always

5. You have a DataFrame df with columns ['product', 'price', 'quantity']. You want to quickly find the average price of products that have quantity greater than 10. Which code snippet correctly performs this first data analysis step?

hard

A. df[df['quantity'] > 10]['price'].mean()

B. df['price'].mean(df['quantity'] > 10)

C. df.mean()['price'] > 10

D. df.describe('price' > 10)

Solution

1. Step 1: Filter rows where quantity > 10

   Use df[df['quantity'] > 10] to select only rows with quantity greater than 10.

2. Step 2: Calculate mean of 'price' column on filtered data

   Access the 'price' column of filtered rows and call mean() to get average price.

3. Final Answer:

   df[df['quantity'] > 10]['price'].mean() -> Option A

4. Quick Check:

   Filter then mean() gives correct average [OK]

Hint: Filter first, then calculate mean [OK]

Common Mistakes:

• Passing condition inside mean() incorrectly
• Using mean() on entire DataFrame without filtering
• Wrong syntax for describe()