query() for fast filtering in Pandas - Step-by-Step Execution

Concept Flow - query() for fast filtering

Start with DataFrame

↓

Write query string

↓

Apply df.query(query_string)

↓

Filter rows matching condition

↓

Return filtered DataFrame

The flow shows how a DataFrame is filtered by writing a query string and applying df.query() to get matching rows.

Execution Sample

Pandas

import pandas as pd

df = pd.DataFrame({
    'age': [25, 30, 35, 40],
    'city': ['NY', 'NY', 'NY', 'SF']
})

filtered = df.query('age > 30 and city == "NY"')

This code filters rows where age is greater than 30 and city is NY using df.query().

Execution Table

Step	Action	Query Condition	Rows Checked	Rows Matching	Resulting DataFrame
1	Start with full DataFrame	age > 30 and city == 'NY'	All rows (4)	N/A	[All 4 rows]
2	Check row 0	age=25, city=NY	Row 0	False (25 > 30? No)	Exclude row 0
3	Check row 1	age=30, city=NY	Row 1	False (30 > 30? No)	Exclude row 1
4	Check row 2	age=35, city=NY	Row 2	True (35 > 30 and city NY)	Include row 2
5	Check row 3	age=40, city=SF	Row 3	False (city SF != NY)	Exclude row 3
6	Return filtered DataFrame	N/A	N/A	Rows matching: 1	DataFrame with row 2 only

💡 All rows checked; only row 2 matches the query condition.

Variable Tracker

Variable	Start	After Step 2	After Step 3	After Step 4	After Step 5	Final
filtered	Empty	Empty	Empty	Contains row 2	Contains row 2	Contains row 2

Key Moments - 2 Insights

Why does df.query('age > 30 and city == "NY"') exclude rows where age is exactly 30?

Because the condition uses '>' which means strictly greater than 30. Rows with age 30 do not satisfy 'age > 30' as shown in execution_table row 3.

How does df.query handle string comparisons like city == 'NY'?

It compares the string values exactly. Only rows where city equals 'NY' are included, as seen in execution_table row 4 where city is 'NY' and row 5 where city is 'SF' is excluded.

Visual Quiz - 3 Questions

Look at the execution table, which row is included in the filtered DataFrame?

A Row 0

B Row 1

C Row 2

D Row 3

💡 Hint

Check the 'Rows Matching' column in execution_table rows 2-5.

At which step does the query condition become false for row 1?

A Step 3

B Step 4

C Step 2

D Step 5

💡 Hint

Look at the 'Rows Checked' and 'Rows Matching' columns for row 1 in execution_table.

If the query was changed to 'age >= 30 and city == "NY"', which additional row would be included?

A Row 0

B Row 1

C Row 3

D Row 1 and Row 3

💡 Hint

Check which rows have age exactly 30 and city 'NY' in variable_tracker and execution_table.

Concept Snapshot

df.query('condition') filters DataFrame rows fast.
Use a string condition like 'age > 30 and city == "NY"'.
It returns rows where condition is True.
Supports logical operators: and, or, not.
Works with column names directly.
Faster and cleaner than boolean indexing.

Full Transcript

This visual execution shows how pandas df.query() filters rows by a condition string. Starting with a DataFrame of ages and cities, the query 'age > 30 and city == "NY"' is applied. Each row is checked: rows 0 and 1 fail because age is not greater than 30, row 3 fails because city is not NY. Only row 2 matches both conditions and is included in the filtered DataFrame. Variables track the filtered result building up. Key moments clarify why exact comparisons matter and how strings are matched. The quiz tests understanding of which rows pass or fail and how changing the condition affects results. The snapshot summarizes usage: df.query() is a clean, fast way to filter DataFrames using readable string conditions.