Boolean indexing in Pandas - Step-by-Step Execution

Concept Flow - Boolean indexing

Start with DataFrame

↓

Create Boolean Condition

↓

Apply Condition to DataFrame

↓

Select Rows where Condition is True

↓

Return Filtered DataFrame

Boolean indexing filters rows in a DataFrame by applying a True/False condition to select only matching rows.

Execution Sample

Pandas

import pandas as pd

df = pd.DataFrame({'Age': [21, 35, 42, 18], 'Name': ['Ann', 'Bob', 'Cara', 'Dan']})

filtered = df[df['Age'] > 30]
print(filtered)

This code filters the DataFrame to keep only rows where Age is greater than 30.

Execution Table

Step	Action	Condition Evaluation	Rows Selected	Output DataFrame
1	Create DataFrame	N/A	All rows	[{'Age':21,'Name':'Ann'}, {'Age':35,'Name':'Bob'}, {'Age':42,'Name':'Cara'}, {'Age':18,'Name':'Dan'}]
2	Evaluate condition df['Age'] > 30	[False, True, True, False]	Rows 1 and 2 (0-based index)	N/A
3	Apply boolean indexing df[df['Age'] > 30]	N/A	Rows 1 and 2	[{'Age':35,'Name':'Bob'}, {'Age':42,'Name':'Cara'}]
4	Print filtered DataFrame	N/A	Rows 1 and 2	Age Name 1 35 Bob 2 42 Cara

💡 All rows evaluated; only rows with Age > 30 selected.

Variable Tracker

Variable	Start	After Step 2	After Step 3	Final
df	[{'Age':21,'Name':'Ann'}, {'Age':35,'Name':'Bob'}, {'Age':42,'Name':'Cara'}, {'Age':18,'Name':'Dan'}]	Same	Same	Same
condition	N/A	[False, True, True, False]	[False, True, True, False]	[False, True, True, False]
filtered	N/A	N/A	[{'Age':35,'Name':'Bob'}, {'Age':42,'Name':'Cara'}]	[{'Age':35,'Name':'Bob'}, {'Age':42,'Name':'Cara'}]

Key Moments - 3 Insights

Why does the filtered DataFrame only show some rows, not all?

Because boolean indexing selects only rows where the condition is True, as shown in execution_table step 3 where only rows with Age > 30 are kept.

What does the condition df['Age'] > 30 return?

It returns a Series of True/False values for each row, indicating if the Age is greater than 30, as shown in execution_table step 2.

Can the original DataFrame 'df' change after filtering?

No, the original DataFrame stays the same; filtering creates a new DataFrame 'filtered' with only selected rows, as shown in variable_tracker.

Visual Quiz - 3 Questions

Look at the execution_table at step 2, what is the condition value for the first row (index 0)?

A True

B None

C False

D Error

💡 Hint

Check the 'Condition Evaluation' column in step 2 of the execution_table.

At which step does the filtered DataFrame get created?

A Step 2

B Step 3

C Step 1

D Step 4

💡 Hint

Look for when boolean indexing is applied to create 'filtered' in the execution_table.

If the condition was changed to df['Age'] > 40, how would the number of selected rows change?

A Decrease to 1 row

B Stay the same at 2 rows

C Increase to 3 rows

D No rows selected

💡 Hint

Refer to the condition evaluation logic and original ages in variable_tracker.

Concept Snapshot

Boolean indexing filters DataFrame rows using a True/False condition.
Syntax: filtered = df[condition]
Condition returns a boolean Series matching DataFrame rows.
Only rows with True condition are kept.
Original DataFrame stays unchanged.
Useful for quick data filtering.

Full Transcript

Boolean indexing in pandas lets you filter rows by applying a condition that returns True or False for each row. You start with a DataFrame, create a condition like df['Age'] > 30, which returns a series of True/False values. Then you use this condition inside df[condition] to select only rows where the condition is True. The result is a new DataFrame with filtered rows. The original DataFrame remains unchanged. This method is simple and powerful for selecting data based on criteria.

Practice

1. What does boolean indexing in pandas do?

easy

A. Changes the data type of columns

B. Sorts the DataFrame by column values

C. Selects rows based on True/False conditions

D. Merges two DataFrames together

Solution

1. Step 1: Understand boolean indexing

   Boolean indexing uses True/False values to filter rows in a DataFrame.

2. Step 2: Identify its purpose

   It selects only rows where the condition is True, filtering data easily.

3. Final Answer:

   Selects rows based on True/False conditions -> Option C

4. Quick Check:

   Boolean indexing = row filtering [OK]

Hint: Boolean indexing filters rows with True conditions [OK]

Common Mistakes:

• Confusing boolean indexing with sorting
• Thinking it changes data types
• Mixing it up with DataFrame merging

2. Which of the following is the correct syntax to select rows where column 'age' is greater than 30 in a DataFrame df?

easy

A. df['age' > 30]

B. df[df['age'] > 30]

C. df[df.age = 30]

D. df[df['age'] => 30]

Solution

1. Step 1: Check correct boolean indexing syntax

   Use df[condition], where condition is df['age'] > 30.

2. Step 2: Identify syntax errors in other options

   df['age' > 30] misplaces brackets, C uses assignment (=) instead of comparison, D uses invalid operator =>.

3. Final Answer:

   df[df['age'] > 30] -> Option B

4. Quick Check:

   Correct syntax uses df[condition] with comparison [OK]

Hint: Use df[condition] with correct comparison operators [OK]

Common Mistakes:

• Using = instead of > for comparison
• Wrong bracket placement
• Using invalid operators like =>

3. Given the DataFrame df below:

   name  age  score
0  Anna   25     88
1  Bob    32     75
2  Cara   29     95
3  Dave   35     60
4  Ella   28     82

What will df[(df['age'] > 28) & (df['score'] > 70)] return?

medium

A. Rows with age > 28 and score > 70: Anna and Ella

B. Rows with age > 28 and score > 70: Bob and Dave

C. Rows with age > 28 and score > 70: Dave only

D. Rows with age > 28 and score > 70: Bob and Cara

Solution

1. Step 1: Identify rows where age > 28

   Rows: Bob (32), Cara (29), Dave (35) satisfy age > 28.

2. Step 2: From those, select rows with score > 70

   Bob (75), Cara (95), Dave (60). Dave fails score > 70, so only Bob and Cara remain.

3. Final Answer:

   Rows with age > 28 and score > 70: Bob and Cara -> Option D

4. Quick Check:

   Filter by both conditions = Bob and Cara [OK]

Hint: Check each condition separately, then combine results [OK]

Common Mistakes:

• Including Dave who has score 60
• Mixing up & and | operators
• Ignoring parentheses around conditions

4. What is wrong with this code snippet for filtering rows where score is above 80?

df[df['score'] > 80 & df['age'] < 30]

medium

A. Missing parentheses around conditions causing operator precedence error

B. Using & instead of | for combining conditions

C. Incorrect column name 'score'

D. DataFrame df is not defined

Solution

1. Step 1: Understand operator precedence in pandas boolean indexing

   Bitwise operators (&, |) require parentheses around each condition to avoid errors.

2. Step 2: Identify missing parentheses

   Code lacks parentheses around df['score'] > 80 and df['age'] < 30, causing wrong evaluation.

3. Final Answer:

   Missing parentheses around conditions causing operator precedence error -> Option A

4. Quick Check:

   Use parentheses with & and | [OK]

Hint: Always wrap each condition in parentheses when using & or | [OK]

Common Mistakes:

• Forgetting parentheses around each condition
• Confusing & with logical and (and)
• Assuming no error without parentheses

5. You have a DataFrame df with columns 'city', 'temperature', and 'humidity'. You want to select rows where city is either 'Paris' or 'London' and temperature is above 20. Which code correctly applies boolean indexing?

hard

A. df[((df['city'] == 'Paris') | (df['city'] == 'London')) & (df['temperature'] > 20)]

B. df[df['city'] == 'Paris' | df['city'] == 'London' & (df['temperature'] > 20)]

C. df[df['city'] == ('Paris' or 'London') & df['temperature'] > 20]

D. df[(df['city'] == 'Paris') | (df['city'] == 'London') & (df['temperature'] > 20)]

Solution

1. Step 1: Understand condition grouping with parentheses

   Conditions combined with | must be wrapped in parentheses to group city checks correctly.

2. Step 2: Check each option for correct syntax and logic

   df[((df['city'] == 'Paris') | (df['city'] == 'London')) & (df['temperature'] > 20)] correctly groups city conditions and combines with temperature condition using &.

3. Final Answer:

   df[((df['city'] == 'Paris') | (df['city'] == 'London')) & (df['temperature'] > 20)] -> Option A

4. Quick Check:

   Group OR conditions with parentheses before AND [OK]

Hint: Group OR conditions in parentheses before AND conditions [OK]

Common Mistakes:

• Missing parentheses causing wrong condition order
• Using Python 'or' instead of |
• Trying to compare city to ('Paris' or 'London') directly