isin() for value matching in Pandas

Introduction

We use isin() to quickly check if values in a list or column match any values from another list. It helps us filter or find data easily.

You want to find all rows in a table where a column has certain values.

You have a list of favorite items and want to see which ones appear in your data.

You want to filter data to only include specific categories or groups.

You want to mark or flag data points that belong to a set of interest.

Syntax

Pandas

DataFrame['column_name'].isin([value1, value2, ...])

This returns a series of True/False showing if each value is in the list.

You can use this result to filter rows or create new columns.

Examples

Check if each fruit in the 'fruit' column is either 'apple' or 'banana'.

Pandas

df['fruit'].isin(['apple', 'banana'])

Filter rows where the 'color' column is 'red' or 'blue'.

Pandas

df[df['color'].isin(['red', 'blue'])]

Create a new column marking if 'item' is in the favorite list.

Pandas

df['is_favorite'] = df['item'].isin(['pen', 'notebook'])

Sample Program

This code creates a small table of people and their cities. Then it finds who lives in New York or Chicago using isin(). Finally, it shows only those people.

Pandas

import pandas as pd

data = {'name': ['Alice', 'Bob', 'Charlie', 'David'],
        'city': ['New York', 'Los Angeles', 'New York', 'Chicago']}
df = pd.DataFrame(data)

# We want to find people who live in New York or Chicago
cities_to_find = ['New York', 'Chicago']

# Use isin() to check
mask = df['city'].isin(cities_to_find)

# Filter the DataFrame
filtered_df = df[mask]

print(filtered_df)

Important Notes

isin() works with lists, sets, or any iterable of values.

It is case-sensitive when matching strings.

You can combine isin() with other conditions for more complex filters.

Summary

isin() helps find if values belong to a list.

It returns True or False for each value checked.

Use it to filter or mark data easily.

Practice

1. What does the isin() function do in pandas?

easy

A. Checks if each value in a Series or DataFrame is in a given list and returns True or False.

B. Sorts the values in a DataFrame or Series.

C. Calculates the sum of values in a DataFrame column.

D. Removes duplicate values from a DataFrame.

Solution

1. Step 1: Understand the purpose of isin()

   isin() checks if each element in a pandas Series or DataFrame exists in a list or another iterable.

2. Step 2: Identify the output type

   The function returns a boolean Series or DataFrame with True where the value is found and False otherwise.

3. Final Answer:

   Checks if each value in a Series or DataFrame is in a given list and returns True or False. -> Option A

4. Quick Check:

   isin() = membership check [OK]

Hint: Remember: isin() returns True/False for membership [OK]

Common Mistakes:

• Thinking isin() sorts data
• Confusing isin() with aggregation functions
• Assuming it removes duplicates

2. Which of the following is the correct syntax to filter rows in a DataFrame df where column col has values in the list [1, 2, 3]?

easy

A. df.isin(col, [1, 2, 3])

B. df.col.isin([1, 2, 3]) = True

C. df[df.isin(col, [1, 2, 3])]

D. df[df.col.isin([1, 2, 3])]

Solution

1. Step 1: Use isin() on the column

   The correct way is to call isin() on the Series: df.col.isin([1, 2, 3]).

2. Step 2: Use boolean indexing to filter rows

   Wrap the boolean Series inside df[ ... ] to select rows where condition is True.

3. Final Answer:

   df[df.col.isin([1, 2, 3])] -> Option D

4. Quick Check:

   Filter rows with boolean mask = df[df.col.isin([1, 2, 3])] [OK]

Hint: Call isin() on column, then filter with df[...] [OK]

Common Mistakes:

• Calling isin() on DataFrame with wrong arguments
• Using assignment instead of filtering
• Incorrect method call syntax

3. Given the DataFrame:

import pandas as pd
df = pd.DataFrame({'A': [10, 20, 30, 40], 'B': ['x', 'y', 'z', 'x']})

What is the output of df['B'].isin(['x', 'z'])?

medium

A. [True, False, True, True]

B. [False, True, False, False]

C. [True, True, True, True]

D. [False, False, False, False]

Solution

1. Step 1: Check each value in column 'B'

   Values are ['x', 'y', 'z', 'x']. We check if each is in ['x', 'z'].

2. Step 2: Mark True if value is in list, else False

   'x' is in list -> True, 'y' not in list -> False, 'z' in list -> True, 'x' in list -> True.

3. Final Answer:

   [True, False, True, True] -> Option A

4. Quick Check:

   Membership check matches [True, False, True, True] [OK]

Hint: Check each value against list, True if found [OK]

Common Mistakes:

• Mixing True/False order
• Confusing column values
• Assuming all True or all False

4. What is wrong with this code snippet?

df = pd.DataFrame({'num': [1, 2, 3, 4]})
mask = df.isin([1, 3])
filtered = df[mask]

medium

A. The DataFrame df must have a column named 'mask' to filter.

B. The isin() method is called on the whole DataFrame, resulting in a DataFrame of booleans, which cannot be used directly for filtering.

C. The variable mask should be converted to a list before filtering.

D. The list passed to isin() must be a string, not integers.

Solution

1. Step 1: Understand df.isin([1, 3]) output

   This returns a DataFrame of booleans with the same shape as df, not a single boolean Series.

2. Step 2: Why filtering with DataFrame mask fails

   Using a DataFrame of booleans as an index mask causes an error because pandas expects a boolean Series or array matching the index length.

3. Final Answer:

   The isin() method is called on the whole DataFrame, resulting in a DataFrame of booleans, which cannot be used directly for filtering. -> Option B

4. Quick Check:

   Mask must be Series, not DataFrame [OK]

Hint: Use isin() on a column, not whole DataFrame for filtering [OK]

Common Mistakes:

• Using DataFrame mask instead of Series mask for filtering
• Passing wrong data types to isin()
• Trying to filter with a DataFrame of booleans

5. You have a DataFrame df with a column status containing values like 'new', 'pending', 'done', and 'cancelled'. You want to create a new column is_active that is True only if status is 'new' or 'pending'. Which code correctly does this using isin()?

hard

A. df['is_active'] = df['status'] == ['new', 'pending']

B. df['is_active'] = df.isin(['new', 'pending'])

C. df['is_active'] = df['status'].isin(['new', 'pending'])

D. df['is_active'] = df['status'].isin('new', 'pending')

Solution

1. Step 1: Use isin() on the 'status' column with a list of target values

   Correct syntax is df['status'].isin(['new', 'pending']) which returns a boolean Series.

2. Step 2: Assign the boolean Series to new column 'is_active'

   This creates a new column with True where status is 'new' or 'pending', False otherwise.

3. Final Answer:

   df['is_active'] = df['status'].isin(['new', 'pending']) -> Option C

4. Quick Check:

   Use isin() on column with list [OK]

Hint: Assign isin() result directly to new column [OK]

Common Mistakes:

• Calling isin() on whole DataFrame
• Passing multiple arguments instead of a list
• Using equality operator with list