Selecting rows by condition in Pandas - Step-by-Step Execution

Concept Flow - Selecting rows by condition

Start with DataFrame

↓

Define condition

↓

Apply condition to DataFrame

↓

Get boolean mask

↓

Filter rows where mask is True

↓

Return filtered DataFrame

We start with a DataFrame, define a condition, apply it to get a mask, then select rows where the condition is true.

Execution Sample

Pandas

import pandas as pd

df = pd.DataFrame({'Age': [22, 35, 58, 45], 'Name': ['Ann', 'Bob', 'Cara', 'Dan']})

filtered = df[df['Age'] > 30]
print(filtered)

This code selects rows where the 'Age' column is greater than 30.

Execution Table

Step	Action	Condition Evaluation	Boolean Mask	Filtered Rows
1	Create DataFrame	N/A	N/A	All rows: Ann(22), Bob(35), Cara(58), Dan(45)
2	Define condition df['Age'] > 30	Age values: [22, 35, 58, 45]	Mask: [False, True, True, True]	N/A
3	Apply mask to df	N/A	Mask: [False, True, True, True]	Rows: Bob(35), Cara(58), Dan(45)
4	Print filtered DataFrame	N/A	N/A	Bob(35), Cara(58), Dan(45)
5	End	N/A	N/A	Filtered DataFrame returned

💡 Filtering ends after applying condition mask; only rows with True mask remain.

Variable Tracker

Variable	Start	After Step 2	After Step 3	Final
df	Empty	Created with 4 rows	Unchanged	Unchanged
condition	Undefined	Evaluated to boolean mask	Unchanged	Unchanged
filtered	Undefined	Undefined	Filtered rows assigned	Filtered DataFrame with 3 rows

Key Moments - 3 Insights

Why does the boolean mask have True or False values?

The mask shows which rows meet the condition (Age > 30). True means the row satisfies the condition and will be kept (see execution_table step 2).

What happens to rows where the condition is False?

Rows with False in the mask are excluded from the filtered DataFrame (see execution_table step 3). They do not appear in the output.

Is the original DataFrame changed after filtering?

No, the original DataFrame stays the same. Filtering creates a new DataFrame with only the selected rows (see variable_tracker for 'df' and 'filtered').

Visual Quiz - 3 Questions

Look at the execution table at step 2, what is the boolean mask for the 'Age' values?

A [True, True, False, False]

B [False, True, True, True]

C [True, False, True, False]

D [False, False, False, True]

💡 Hint

Check the 'Boolean Mask' column at step 2 in the execution_table.

At which step do we get the filtered rows with only ages above 30?

A Step 3

B Step 2

C Step 1

D Step 4

💡 Hint

Look at the 'Filtered Rows' column in the execution_table to find when filtering happens.

If we change the condition to df['Age'] > 50, how would the boolean mask change at step 2?

A [True, True, True, True]

B [False, True, False, True]

C [False, False, True, False]

D [True, False, False, False]

💡 Hint

Think about which ages are greater than 50 and how that affects the mask in execution_table step 2.

Concept Snapshot

Selecting rows by condition in pandas:
Use df[condition] where condition is a boolean expression on columns.
This returns a new DataFrame with rows where condition is True.
Original DataFrame stays unchanged.
Example: df[df['Age'] > 30] filters rows with Age > 30.

Full Transcript

We start with a DataFrame containing names and ages. We define a condition to select rows where the Age is greater than 30. Applying this condition creates a boolean mask that marks True for rows meeting the condition and False otherwise. Using this mask to filter the DataFrame returns only the rows where the mask is True. The original DataFrame remains unchanged. This process helps us select specific rows based on any condition we want.

Practice

1. What does the expression df[df['age'] > 30] do in pandas?

easy

A. Deletes rows where the 'age' column is greater than 30

B. Selects columns where the 'age' value is greater than 30

C. Selects rows where the 'age' column is greater than 30

D. Sorts the DataFrame by the 'age' column

Solution

1. Step 1: Understand the condition inside the brackets

   The condition df['age'] > 30 creates a True/False mask for each row based on the 'age' value.

2. Step 2: Apply the condition to select rows

   Using df[condition] returns only rows where the condition is True, i.e., 'age' > 30.

3. Final Answer:

   Selects rows where the 'age' column is greater than 30 -> Option C

4. Quick Check:

   Condition filters rows [OK]

Hint: Use df[condition] to filter rows by True/False mask [OK]

Common Mistakes:

• Thinking it selects columns instead of rows
• Confusing filtering with sorting
• Using parentheses incorrectly around conditions

2. Which of the following is the correct syntax to select rows where column 'score' equals 100?

easy

A. df[df.score = 100]

B. df[df['score'] = 100]

C. df[df['score'] === 100]

D. df[df['score'] == 100]

Solution

1. Step 1: Identify correct equality operator in pandas

   In pandas, to compare values, use double equals ==, not single = or triple ===.

2. Step 2: Check correct column access syntax

   Using df['score'] is correct and safe for all column names.

3. Final Answer:

   df[df['score'] == 100] -> Option D

4. Quick Check:

   Use == for comparison and brackets for column [OK]

Hint: Use df[df['col'] == value] with double equals for condition [OK]

Common Mistakes:

• Using single = instead of == for comparison
• Using triple === which is invalid in Python
• Accessing columns without quotes when name has special chars

3. Given the DataFrame:

import pandas as pd
data = {'name': ['Ann', 'Bob', 'Cara', 'Dan'], 'age': [25, 35, 30, 40]}
df = pd.DataFrame(data)

What is the output of df[df['age'] >= 30]?

medium

A. Rows with names Bob, Cara, and Dan

B. Rows with names Ann and Bob

C. Rows with names Ann and Cara

D. Rows with names Dan only

Solution

1. Step 1: Evaluate the condition df['age'] >= 30

   This condition is True for ages 35 (Bob), 30 (Cara), and 40 (Dan).

2. Step 2: Select rows where condition is True

   Rows with names Bob, Cara, and Dan are returned.

3. Final Answer:

   Rows with names Bob, Cara, and Dan -> Option A

4. Quick Check:

   age >= 30 selects Bob, Cara, Dan [OK]

Hint: Check condition True rows, then list their names [OK]

Common Mistakes:

• Including Ann who is 25 (less than 30)
• Confusing >= with >
• Misreading the DataFrame data

4. What is wrong with this code to select rows where 'score' is above 50?

df[df['score'] > 50 && df['age'] < 30]

medium

A. Using && instead of & for combining conditions

B. Missing parentheses around each condition

C. Using single & instead of &&

D. Using | instead of & for and condition

Solution

1. Step 1: Check operator for combining conditions in pandas

   In pandas, use single ampersand & for 'and', not double ampersand &&.

2. Step 2: Confirm correct syntax

   Correct syntax is df[(df['score'] > 50) & (df['age'] < 30)] with parentheses and single &.

3. Final Answer:

   Using && instead of & for combining conditions -> Option A

4. Quick Check:

   Use & not && for and in pandas [OK]

Hint: Use & for and, | for or, with parentheses [OK]

Common Mistakes:

• Using && which causes syntax error
• Not adding parentheses around each condition
• Using | instead of & for and

5. You have a DataFrame df with columns 'city', 'temperature', and 'humidity'. How do you select rows where the city is 'Paris' and temperature is above 20 or humidity is below 30?

hard

A. df[(df['city'] == 'Paris') & (df['temperature'] > 20) | (df['humidity'] < 30)]

B. df[(df['city'] == 'Paris') & ((df['temperature'] > 20) | (df['humidity'] < 30))]

C. df[df['city'] == 'Paris' & df['temperature'] > 20 | df['humidity'] < 30]

D. df[df['city'] == 'Paris' and df['temperature'] > 20 or df['humidity'] < 30]

Solution

1. Step 1: Understand operator precedence and grouping

   Logical AND (&) has higher precedence than OR (|), so parentheses are needed to group OR conditions correctly.

2. Step 2: Write correct condition with parentheses

   We want city == 'Paris' AND (temperature > 20 OR humidity < 30), so use:
   df[(df['city'] == 'Paris') & ((df['temperature'] > 20) | (df['humidity'] < 30))]

3. Final Answer:

   df[(df['city'] == 'Paris') & ((df['temperature'] > 20) | (df['humidity'] < 30))] -> Option B

4. Quick Check:

   Group OR inside AND with parentheses [OK]

Hint: Use parentheses to group OR inside AND conditions [OK]

Common Mistakes:

• Missing parentheses causing wrong logic
• Using Python 'and'/'or' instead of & and |
• Not using parentheses around each condition