Selecting rows by condition in Pandas - Time & Space Complexity

Time Complexity: Selecting rows by condition

O(n)

Understanding Time Complexity

We want to know how the time to select rows by a condition changes as the data grows.

How does the work increase when the table gets bigger?

Scenario Under Consideration

Analyze the time complexity of the following code snippet.

import pandas as pd

df = pd.DataFrame({'age': [23, 45, 12, 36, 27] * 200})

selected = df[df['age'] > 30]

This code creates a table with ages and selects rows where age is greater than 30.

Identify Repeating Operations

• Primary operation: Checking each row's 'age' value against 30.
• How many times: Once for every row in the table.

How Execution Grows With Input

As the number of rows grows, the time to check each row grows at the same rate.

Input Size (n)	Approx. Operations
10	10 checks
100	100 checks
1000	1000 checks

Pattern observation: The work grows directly with the number of rows.

Final Time Complexity

Time Complexity: O(n)

This means the time to select rows grows in a straight line as the table gets bigger.

Common Mistake

[X] Wrong: "Selecting rows by condition is instant no matter the size."

[OK] Correct: The code must check each row to see if it meets the condition, so more rows mean more work.

Interview Connect

Understanding how filtering data scales helps you write efficient code and explain your choices clearly.

Self-Check

"What if we select rows using multiple conditions combined with AND? How would the time complexity change?"

Practice

1. What does the expression df[df['age'] > 30] do in pandas?

easy

A. Deletes rows where the 'age' column is greater than 30

B. Selects columns where the 'age' value is greater than 30

C. Selects rows where the 'age' column is greater than 30

D. Sorts the DataFrame by the 'age' column

Solution

1. Step 1: Understand the condition inside the brackets

   The condition df['age'] > 30 creates a True/False mask for each row based on the 'age' value.

2. Step 2: Apply the condition to select rows

   Using df[condition] returns only rows where the condition is True, i.e., 'age' > 30.

3. Final Answer:

   Selects rows where the 'age' column is greater than 30 -> Option C

4. Quick Check:

   Condition filters rows [OK]

Hint: Use df[condition] to filter rows by True/False mask [OK]

Common Mistakes:

• Thinking it selects columns instead of rows
• Confusing filtering with sorting
• Using parentheses incorrectly around conditions

2. Which of the following is the correct syntax to select rows where column 'score' equals 100?

easy

A. df[df.score = 100]

B. df[df['score'] = 100]

C. df[df['score'] === 100]

D. df[df['score'] == 100]

Solution

1. Step 1: Identify correct equality operator in pandas

   In pandas, to compare values, use double equals ==, not single = or triple ===.

2. Step 2: Check correct column access syntax

   Using df['score'] is correct and safe for all column names.

3. Final Answer:

   df[df['score'] == 100] -> Option D

4. Quick Check:

   Use == for comparison and brackets for column [OK]

Hint: Use df[df['col'] == value] with double equals for condition [OK]

Common Mistakes:

• Using single = instead of == for comparison
• Using triple === which is invalid in Python
• Accessing columns without quotes when name has special chars

3. Given the DataFrame:

import pandas as pd
data = {'name': ['Ann', 'Bob', 'Cara', 'Dan'], 'age': [25, 35, 30, 40]}
df = pd.DataFrame(data)

What is the output of df[df['age'] >= 30]?

medium

A. Rows with names Bob, Cara, and Dan

B. Rows with names Ann and Bob

C. Rows with names Ann and Cara

D. Rows with names Dan only

Solution

1. Step 1: Evaluate the condition df['age'] >= 30

   This condition is True for ages 35 (Bob), 30 (Cara), and 40 (Dan).

2. Step 2: Select rows where condition is True

   Rows with names Bob, Cara, and Dan are returned.

3. Final Answer:

   Rows with names Bob, Cara, and Dan -> Option A

4. Quick Check:

   age >= 30 selects Bob, Cara, Dan [OK]

Hint: Check condition True rows, then list their names [OK]

Common Mistakes:

• Including Ann who is 25 (less than 30)
• Confusing >= with >
• Misreading the DataFrame data

4. What is wrong with this code to select rows where 'score' is above 50?

df[df['score'] > 50 && df['age'] < 30]

medium

A. Using && instead of & for combining conditions

B. Missing parentheses around each condition

C. Using single & instead of &&

D. Using | instead of & for and condition

Solution

1. Step 1: Check operator for combining conditions in pandas

   In pandas, use single ampersand & for 'and', not double ampersand &&.

2. Step 2: Confirm correct syntax

   Correct syntax is df[(df['score'] > 50) & (df['age'] < 30)] with parentheses and single &.

3. Final Answer:

   Using && instead of & for combining conditions -> Option A

4. Quick Check:

   Use & not && for and in pandas [OK]

Hint: Use & for and, | for or, with parentheses [OK]

Common Mistakes:

• Using && which causes syntax error
• Not adding parentheses around each condition
• Using | instead of & for and

5. You have a DataFrame df with columns 'city', 'temperature', and 'humidity'. How do you select rows where the city is 'Paris' and temperature is above 20 or humidity is below 30?

hard

A. df[(df['city'] == 'Paris') & (df['temperature'] > 20) | (df['humidity'] < 30)]

B. df[(df['city'] == 'Paris') & ((df['temperature'] > 20) | (df['humidity'] < 30))]

C. df[df['city'] == 'Paris' & df['temperature'] > 20 | df['humidity'] < 30]

D. df[df['city'] == 'Paris' and df['temperature'] > 20 or df['humidity'] < 30]

Solution

1. Step 1: Understand operator precedence and grouping

   Logical AND (&) has higher precedence than OR (|), so parentheses are needed to group OR conditions correctly.

2. Step 2: Write correct condition with parentheses

   We want city == 'Paris' AND (temperature > 20 OR humidity < 30), so use:
   df[(df['city'] == 'Paris') & ((df['temperature'] > 20) | (df['humidity'] < 30))]

3. Final Answer:

   df[(df['city'] == 'Paris') & ((df['temperature'] > 20) | (df['humidity'] < 30))] -> Option B

4. Quick Check:

   Group OR inside AND with parentheses [OK]

Hint: Use parentheses to group OR inside AND conditions [OK]

Common Mistakes:

• Missing parentheses causing wrong logic
• Using Python 'and'/'or' instead of & and |
• Not using parentheses around each condition