Why exploratory inspection guides analysis in Data Analysis Python - Performance Analysis

Time Complexity: Why exploratory inspection guides analysis

O(c * n)

Understanding Time Complexity

Exploratory inspection means looking closely at data before deep analysis.

We want to know how this step affects the time it takes to analyze data.

Scenario Under Consideration

Analyze the time complexity of the following code snippet.

import pandas as pd

def inspect_data(df):
    print(df.head())
    print(df.describe())
    for col in df.columns:
        print(f"Unique values in {col}:", df[col].nunique())

This code prints the first rows, summary stats, and counts unique values per column.

Identify Repeating Operations

Identify the loops, recursion, array traversals that repeat.

• Primary operation: Loop over each column to count unique values.
• How many times: Once per column in the data frame.

How Execution Grows With Input

As the number of columns grows, the unique count runs more times.

Input Size (columns)	Approx. Operations
10	10 unique counts
100	100 unique counts
1000	1000 unique counts

Pattern observation: The work grows directly with the number of columns.

Final Time Complexity

Time Complexity: O(c * n)

This means the time grows with columns (c) times rows (n) because counting unique values checks all rows per column.

Common Mistake

[X] Wrong: "Exploratory inspection is always fast and does not affect analysis time."

[OK] Correct: Counting unique values or summaries can take time proportional to data size, so inspection can be costly on big data.

Interview Connect

Understanding how initial data checks scale helps you plan analysis steps wisely and shows you think about efficiency.

Self-Check

"What if we only inspected a random sample of rows instead of the full data? How would the time complexity change?"

Practice

1. Why is exploratory inspection important before starting data analysis?

easy

A. It helps understand data quality and guides cleaning steps.

B. It automatically fixes all data errors.

C. It replaces the need for any analysis.

D. It only shows the last rows of data.

Solution

1. Step 1: Understand the purpose of exploratory inspection

   Exploratory inspection helps us see what the data looks like and if there are any problems.

2. Step 2: Connect inspection to analysis planning

   Knowing data quality helps decide how to clean and analyze the data properly.

3. Final Answer:

   It helps understand data quality and guides cleaning steps. -> Option A

4. Quick Check:

   Exploratory inspection guides analysis = B [OK]

Hint: Exploratory inspection reveals data issues early [OK]

Common Mistakes:

• Thinking inspection fixes errors automatically
• Believing inspection replaces analysis
• Confusing head() with tail()

2. Which of the following Python commands is correct to quickly see the first 5 rows of a DataFrame named df?

easy

A. df.show()

B. df.first5()

C. df.top()

D. df.head()

Solution

1. Step 1: Recall the method to view first rows

   The standard method to view the first 5 rows in pandas is head().

2. Step 2: Check other options for validity

   Methods like first5(), top(), and show() do not exist in pandas DataFrame.

3. Final Answer:

   df.head() -> Option D

4. Quick Check:

   df.head() shows first rows [OK]

Hint: Use df.head() to preview first rows [OK]

Common Mistakes:

• Using non-existent methods like df.top()
• Confusing df.head() with df.tail()
• Forgetting parentheses after method name

3. Given the code:

import pandas as pd
data = {'age': [25, 30, 22, 40], 'income': [50000, 60000, 45000, 80000]}
df = pd.DataFrame(data)
print(df.describe())

What will be the output?

medium

A. Summary statistics including count, mean, std, min, 25%, 50%, 75%, max for each column.

B. Only the first 5 rows of the DataFrame.

C. Data types of each column.

D. An error because describe() needs parameters.

Solution

1. Step 1: Understand what df.describe() does

   The describe() method gives summary statistics like count, mean, std, min, quartiles, and max for numeric columns.

2. Step 2: Check the code context

   The DataFrame has numeric columns 'age' and 'income', so describe() will output their statistics.

3. Final Answer:

   Summary statistics including count, mean, std, min, 25%, 50%, 75%, max for each column. -> Option A

4. Quick Check:

   df.describe() = summary stats [OK]

Hint: describe() summarizes numeric columns [OK]

Common Mistakes:

• Thinking describe() shows data rows
• Expecting data types from describe()
• Assuming describe() needs arguments

4. What is wrong with this code snippet for inspecting a DataFrame df?

print(df.info)

medium

A. print() cannot be used with info().

B. Missing parentheses to call the info() method.

C. info() is not a valid method for DataFrame.

D. df.info() returns data, not info.

Solution

1. Step 1: Identify the method call syntax error

   The code uses df.info without parentheses, so it refers to the method object, not calls it.

2. Step 2: Explain correct usage

   To execute and print the info, it should be print(df.info()) with parentheses.

3. Final Answer:

   Missing parentheses to call the info() method. -> Option B

4. Quick Check:

   Method call needs () [OK]

Hint: Always add () to call methods like info() [OK]

Common Mistakes:

• Forgetting parentheses on method calls
• Thinking info() is not a DataFrame method
• Trying to print method object instead of output

5. You have a DataFrame df with columns ['name', 'age', 'salary']. You want to check if there are missing values before analysis. Which code snippet correctly helps you decide the next steps?

hard

A. print(df.columns)

B. print(df.head(10))

C. print(df.isnull().sum())

D. print(df.describe())

Solution

1. Step 1: Identify method to check missing values

   The method isnull() returns True for missing values; summing counts them per column.

2. Step 2: Understand why other options don't help

   head() shows rows but not missing counts, describe() summarizes numeric data, and columns lists column names only.

3. Final Answer:

   print(df.isnull().sum()) -> Option C

4. Quick Check:

   Use isnull().sum() to count missing data [OK]

Hint: Use isnull().sum() to find missing values fast [OK]

Common Mistakes:

• Using head() to find missing data
• Expecting describe() to show missing counts
• Confusing columns attribute with data content