Boolean indexing in Data Analysis Python - Step-by-Step Execution

Concept Flow - Boolean indexing

Start with DataFrame or Array

↓

Create Boolean Condition

↓

Apply Condition to Data

↓

Select Rows/Elements where Condition is True

↓

Return Filtered Data

Boolean indexing filters data by selecting only elements where a condition is true.

Execution Sample

Data Analysis Python

import pandas as pd

df = pd.DataFrame({'A': [1, 2, 3, 4], 'B': [5, 6, 7, 8]})
filtered = df[df['A'] > 2]
print(filtered)

This code selects rows from the DataFrame where column 'A' has values greater than 2.

Execution Table

Step	Action	Condition Evaluation	Resulting Boolean Array	Filtered DataFrame
1	Create DataFrame	N/A	N/A	A: [1, 2, 3, 4], B: [5, 6, 7, 8]
2	Evaluate condition df['A'] > 2	Check each element in 'A'	[False, False, True, True]	N/A
3	Apply boolean array to df	Select rows where True	N/A	Rows with A=3,B=7 and A=4,B=8
4	Print filtered DataFrame	N/A	N/A	A B 2 3 7 3 4 8

💡 All rows checked; only rows with A > 2 selected.

Variable Tracker

Variable	Start	After Step 2	After Step 3	Final
df	{'A':[1,2,3,4],'B':[5,6,7,8]}	{'A':[1,2,3,4],'B':[5,6,7,8]}	{'A':[1,2,3,4],'B':[5,6,7,8]}	{'A':[1,2,3,4],'B':[5,6,7,8]}
condition	N/A	[False, False, True, True]	[False, False, True, True]	[False, False, True, True]
filtered	N/A	N/A	Rows where condition True	Rows with A=3,B=7 and A=4,B=8

Key Moments - 3 Insights

Why does the filtered DataFrame only show rows with A values greater than 2?

Because the boolean array [False, False, True, True] selects only rows where the condition df['A'] > 2 is True, as shown in execution_table step 3.

What happens if the condition returns all False values?

No rows are selected, resulting in an empty DataFrame. This is because boolean indexing only keeps rows where the condition is True.

Can boolean indexing be used with arrays other than DataFrames?

Yes, boolean indexing works with NumPy arrays and pandas Series similarly by selecting elements where the condition is True.

Visual Quiz - 3 Questions

Look at the execution table, what is the boolean array after evaluating df['A'] > 2?

A [False, False, True, True]

B [True, False, True, False]

C [True, True, False, False]

D [False, True, False, True]

💡 Hint

Check execution_table row 2 under 'Resulting Boolean Array'

At which step is the filtered DataFrame created?

A Step 1

B Step 3

C Step 2

D Step 4

💡 Hint

Look at execution_table rows and see when filtering happens

If the condition was df['A'] > 5, how would the filtered DataFrame change?

A It would include all rows

B It would include only rows where A is 5

C It would be empty

D It would include rows where A is less than 5

💡 Hint

Refer to key_moments about what happens if condition is all False

Concept Snapshot

Boolean indexing filters data by using a condition that returns True or False for each element.
Syntax: filtered = data[condition]
Only elements where condition is True are kept.
Works with pandas DataFrames, Series, and NumPy arrays.
Useful for quick data filtering without loops.

Full Transcript

Boolean indexing is a way to select data by applying a condition that returns True or False for each element. We start with a DataFrame, create a condition like df['A'] > 2, which checks each value in column 'A'. This condition produces a boolean array showing True where the condition holds and False otherwise. Applying this boolean array to the DataFrame selects only the rows where the condition is True. The result is a filtered DataFrame with only those rows. This method works similarly for arrays and Series. If the condition is all False, the result is an empty DataFrame. Boolean indexing is a simple and powerful way to filter data without writing loops.

Practice

1. What does Boolean indexing do in data analysis?

easy

A. Selects data based on true or false conditions

B. Sorts data in ascending order

C. Converts data types automatically

D. Merges two datasets together

Solution

1. Step 1: Understand the purpose of Boolean indexing

   Boolean indexing uses true/false values to filter data.

2. Step 2: Compare with other options

   Sorting, converting types, and merging are different operations.

3. Final Answer:

   Selects data based on true or false conditions -> Option A

4. Quick Check:

   Boolean indexing = filter by true/false [OK]

Hint: Boolean indexing filters data by true/false values [OK]

Common Mistakes:

• Confusing Boolean indexing with sorting
• Thinking it changes data types
• Mixing it up with merging datasets

2. Which of the following is the correct syntax to select rows where column 'age' is greater than 30 in a DataFrame df?

easy

A. df['age' > 30]

B. df[df.age = 30]

C. df[df['age'] > 30]

D. df[df['age'] < 30]

Solution

1. Step 1: Identify correct Boolean indexing syntax

   Use df[condition], where condition is df['age'] > 30.

2. Step 2: Check other options for errors

   df['age' > 30] has wrong brackets, C uses assignment (=) instead of comparison, D uses wrong comparison sign.

3. Final Answer:

   df[df['age'] > 30] -> Option C

4. Quick Check:

   Correct syntax uses df[condition] with comparison [OK]

Hint: Use df[condition] with comparison operators for filtering [OK]

Common Mistakes:

• Using assignment '=' instead of comparison '>'
• Wrong bracket placement
• Using incorrect comparison signs

3. Given the DataFrame df below:

import pandas as pd
data = {'score': [85, 42, 78, 90, 55]}
df = pd.DataFrame(data)

What is the output of df[df['score'] >= 78]?

medium

A. Rows with scores 85, 78, and 90

B. Rows with scores 42 and 55

C. All rows because condition is ignored

D. Empty DataFrame

Solution

1. Step 1: Apply the condition df['score'] >= 78

   This selects rows where score is 78 or more: 85, 78, 90.

2. Step 2: Check which rows match

   Rows with scores 42 and 55 do not match, so they are excluded.

3. Final Answer:

   Rows with scores 85, 78, and 90 -> Option A

4. Quick Check:

   Filter keeps scores >= 78 [OK]

Hint: Filter keeps rows where condition is true [OK]

Common Mistakes:

• Including rows that don't meet the condition
• Thinking condition is ignored
• Expecting empty output incorrectly

4. What is wrong with this code snippet?

import pandas as pd
data = {'value': [10, 20, 30, 40]}
df = pd.DataFrame(data)
filtered = df[df.value > 25]

medium

A. It will raise a syntax error

B. It will raise an AttributeError

C. It returns an empty DataFrame

D. It works correctly and filters rows with value > 25

Solution

1. Step 1: Check attribute access for column 'value'

   Using df.value is valid if column name has no spaces or special chars.

2. Step 2: Verify filtering condition

   df.value > 25 selects rows with values 30 and 40 correctly.

3. Final Answer:

   It works correctly and filters rows with value > 25 -> Option D

4. Quick Check:

   Attribute access works for simple column names [OK]

Hint: Use df.column_name if no spaces or special chars [OK]

Common Mistakes:

• Assuming df.value always causes error
• Expecting syntax error
• Thinking it returns empty DataFrame

5. You have a DataFrame df with a column 'status' containing values 'pass', 'fail', and 'pending'. How do you select only rows where 'status' is either 'pass' or 'pending' using Boolean indexing?

hard

A. df[df['status'] == 'pass' and 'pending']

B. df[(df['status'] == 'pass') | (df['status'] == 'pending')]

C. df[df['status'] in ['pass', 'pending']]

D. df[df['status'] == 'pass' or df['status'] == 'pending']

Solution

1. Step 1: Use element-wise OR for multiple conditions

   Use | between conditions inside parentheses for correct Boolean indexing.

2. Step 2: Avoid Python keywords and wrong syntax

   'and' and 'or' do not work element-wise; 'in' is not vectorized here.

3. Final Answer:

   df[(df['status'] == 'pass') | (df['status'] == 'pending')] -> Option B

4. Quick Check:

   Use | with parentheses for multiple conditions [OK]

Hint: Use | with parentheses for multiple conditions [OK]

Common Mistakes:

• Using 'and' or 'or' instead of | and &
• Trying to use 'in' directly on Series
• Missing parentheses around conditions