Selecting rows by condition in Pandas

Introduction

We select rows by condition to find and work with only the data that matters to us.

You want to see all customers who spent more than $100.

You need to find all students who scored above 90 in a test.

You want to filter products that are in stock.

You want to analyze only data from a specific city.

You want to check records where a value is missing or zero.

Syntax

Pandas

filtered_df = df[df['column_name'] <condition> value]

Use square brackets [] to filter rows based on a condition.

The condition inside the brackets returns True or False for each row.

Examples

Selects rows where the 'age' column is greater than 30.

Pandas

filtered_df = df[df['age'] > 30]

Selects rows where the 'city' column is exactly 'New York'.

Pandas

filtered_df = df[df['city'] == 'New York']

Selects rows where 'score' is at least 80 and 'passed' is True.

Pandas

filtered_df = df[(df['score'] >= 80) & (df['passed'] == True)]

Selects rows where the 'name' column contains the text 'John'.

Pandas

filtered_df = df[df['name'].str.contains('John')]

Sample Program

This code creates a small table of people with their ages and cities. Then it selects only those people older than 30 and prints them.

Pandas

import pandas as pd

data = {'name': ['Alice', 'Bob', 'Charlie', 'David'],
        'age': [25, 35, 30, 40],
        'city': ['New York', 'Los Angeles', 'New York', 'Chicago']}

df = pd.DataFrame(data)

# Select rows where age is greater than 30
filtered_df = df[df['age'] > 30]

print(filtered_df)

Important Notes

Use & for AND and | for OR when combining multiple conditions. Remember to put each condition in parentheses.

String conditions can use methods like str.contains() for partial matches.

Filtering does not change the original data unless you assign it back.

Summary

Use square brackets with a condition to select rows.

Conditions return True or False for each row to keep or drop it.

You can combine multiple conditions with & (and) or | (or).

Practice

1. What does the expression df[df['age'] > 30] do in pandas?

easy

A. Deletes rows where the 'age' column is greater than 30

B. Selects columns where the 'age' value is greater than 30

C. Selects rows where the 'age' column is greater than 30

D. Sorts the DataFrame by the 'age' column

Solution

1. Step 1: Understand the condition inside the brackets

   The condition df['age'] > 30 creates a True/False mask for each row based on the 'age' value.

2. Step 2: Apply the condition to select rows

   Using df[condition] returns only rows where the condition is True, i.e., 'age' > 30.

3. Final Answer:

   Selects rows where the 'age' column is greater than 30 -> Option C

4. Quick Check:

   Condition filters rows [OK]

Hint: Use df[condition] to filter rows by True/False mask [OK]

Common Mistakes:

• Thinking it selects columns instead of rows
• Confusing filtering with sorting
• Using parentheses incorrectly around conditions

2. Which of the following is the correct syntax to select rows where column 'score' equals 100?

easy

A. df[df.score = 100]

B. df[df['score'] = 100]

C. df[df['score'] === 100]

D. df[df['score'] == 100]

Solution

1. Step 1: Identify correct equality operator in pandas

   In pandas, to compare values, use double equals ==, not single = or triple ===.

2. Step 2: Check correct column access syntax

   Using df['score'] is correct and safe for all column names.

3. Final Answer:

   df[df['score'] == 100] -> Option D

4. Quick Check:

   Use == for comparison and brackets for column [OK]

Hint: Use df[df['col'] == value] with double equals for condition [OK]

Common Mistakes:

• Using single = instead of == for comparison
• Using triple === which is invalid in Python
• Accessing columns without quotes when name has special chars

3. Given the DataFrame:

import pandas as pd
data = {'name': ['Ann', 'Bob', 'Cara', 'Dan'], 'age': [25, 35, 30, 40]}
df = pd.DataFrame(data)

What is the output of df[df['age'] >= 30]?

medium

A. Rows with names Bob, Cara, and Dan

B. Rows with names Ann and Bob

C. Rows with names Ann and Cara

D. Rows with names Dan only

Solution

1. Step 1: Evaluate the condition df['age'] >= 30

   This condition is True for ages 35 (Bob), 30 (Cara), and 40 (Dan).

2. Step 2: Select rows where condition is True

   Rows with names Bob, Cara, and Dan are returned.

3. Final Answer:

   Rows with names Bob, Cara, and Dan -> Option A

4. Quick Check:

   age >= 30 selects Bob, Cara, Dan [OK]

Hint: Check condition True rows, then list their names [OK]

Common Mistakes:

• Including Ann who is 25 (less than 30)
• Confusing >= with >
• Misreading the DataFrame data

4. What is wrong with this code to select rows where 'score' is above 50?

df[df['score'] > 50 && df['age'] < 30]

medium

A. Using && instead of & for combining conditions

B. Missing parentheses around each condition

C. Using single & instead of &&

D. Using | instead of & for and condition

Solution

1. Step 1: Check operator for combining conditions in pandas

   In pandas, use single ampersand & for 'and', not double ampersand &&.

2. Step 2: Confirm correct syntax

   Correct syntax is df[(df['score'] > 50) & (df['age'] < 30)] with parentheses and single &.

3. Final Answer:

   Using && instead of & for combining conditions -> Option A

4. Quick Check:

   Use & not && for and in pandas [OK]

Hint: Use & for and, | for or, with parentheses [OK]

Common Mistakes:

• Using && which causes syntax error
• Not adding parentheses around each condition
• Using | instead of & for and

5. You have a DataFrame df with columns 'city', 'temperature', and 'humidity'. How do you select rows where the city is 'Paris' and temperature is above 20 or humidity is below 30?

hard

A. df[(df['city'] == 'Paris') & (df['temperature'] > 20) | (df['humidity'] < 30)]

B. df[(df['city'] == 'Paris') & ((df['temperature'] > 20) | (df['humidity'] < 30))]

C. df[df['city'] == 'Paris' & df['temperature'] > 20 | df['humidity'] < 30]

D. df[df['city'] == 'Paris' and df['temperature'] > 20 or df['humidity'] < 30]

Solution

1. Step 1: Understand operator precedence and grouping

   Logical AND (&) has higher precedence than OR (|), so parentheses are needed to group OR conditions correctly.

2. Step 2: Write correct condition with parentheses

   We want city == 'Paris' AND (temperature > 20 OR humidity < 30), so use:
   df[(df['city'] == 'Paris') & ((df['temperature'] > 20) | (df['humidity'] < 30))]

3. Final Answer:

   df[(df['city'] == 'Paris') & ((df['temperature'] > 20) | (df['humidity'] < 30))] -> Option B

4. Quick Check:

   Group OR inside AND with parentheses [OK]

Hint: Use parentheses to group OR inside AND conditions [OK]

Common Mistakes:

• Missing parentheses causing wrong logic
• Using Python 'and'/'or' instead of & and |
• Not using parentheses around each condition