Pattern matching with str.contains in Data Analysis Python

Introduction

We use pattern matching with str.contains to find rows in data that have certain words or patterns. It helps us quickly filter data based on text.

You want to find all customer reviews that mention the word 'good'.

You need to filter a list of emails to find those containing 'gmail.com'.

You want to select rows where a product description includes 'organic'.

You want to check if any names in a list contain the letter 'a'.

Syntax

Data Analysis Python

import pandas as pd

data = pd.Series(['apple', 'banana', 'cherry', 'date'])

mask = data.str.contains('an')
filtered_data = data[mask]

The str.contains method returns a boolean series showing True where the pattern is found.

You can use regular expressions for complex patterns or simple strings for exact matches.

Examples

This finds all fruits with 'an' in their name.

Data Analysis Python

import pandas as pd

data = pd.Series(['apple', 'banana', 'cherry', 'date'])

# Find items containing 'an'
mask = data.str.contains('an')
print(data[mask])

When the data is empty, the result is also empty without error.

Data Analysis Python

import pandas as pd

data = pd.Series(['apple', 'banana', 'cherry', 'date'])

# What if the series is empty?
empty_data = pd.Series([], dtype=object)
mask_empty = empty_data.str.contains('a')
print(empty_data[mask_empty])

Works fine even with one element.

Data Analysis Python

import pandas as pd

data = pd.Series(['apple'])

# Only one element, check if it contains 'a'
mask_single = data.str.contains('a')
print(data[mask_single])

Using regex to find elements starting with 'a'.

Data Analysis Python

import pandas as pd

data = pd.Series(['apple', 'banana', 'cherry', 'date'])

# Check if elements start with 'a' using regex
mask_start_a = data.str.contains('^a')
print(data[mask_start_a])

Sample Program

This program shows the original list of fruits and then filters to show only those containing the letter 'a'.

Data Analysis Python

import pandas as pd

# Create a Series of fruit names
fruit_names = pd.Series(['apple', 'banana', 'cherry', 'date', 'avocado'])

print('Original data:')
print(fruit_names)

# Find fruits containing the letter 'a'
contains_a_mask = fruit_names.str.contains('a')
filtered_fruits = fruit_names[contains_a_mask]

print('\nFruits containing the letter "a":')
print(filtered_fruits)

Important Notes

Time complexity: O(n) where n is the number of elements, because it checks each string once.

Space complexity: O(n) for the boolean mask created.

Common mistake: Forgetting that str.contains returns a boolean series, so you must use it to filter the original data.

Use str.contains when you want to filter data by text patterns. For exact matches, consider using == or str.match for full string matches.

Summary

Pattern matching with str.contains helps find text inside data easily.

It returns True or False for each item, so you can filter your data.

You can use simple words or complex patterns with regular expressions.

Practice

1. What does the str.contains method do when used on a pandas Series?

easy

A. It replaces all occurrences of a pattern with another string.

B. It counts how many times a pattern appears in each string.

C. It splits each string into a list based on a pattern.

D. It checks if each string contains a specified pattern and returns true or false.

Solution

1. Step 1: Understand str.contains purpose

   The method checks each string in a Series for a pattern and returns true or false accordingly.

2. Step 2: Compare with other string methods

   Replacing, splitting, or counting are different methods like replace(), split(), or count(), not contains().

3. Final Answer:

   It checks if each string contains a specified pattern and returns true or false. -> Option D

4. Quick Check:

   str.contains returns boolean mask [OK]

Hint: Remember: contains returns true/false for pattern presence [OK]

Common Mistakes:

• Confusing contains with replace or split
• Thinking it returns counts instead of booleans
• Assuming it modifies the original data

2. Which of the following is the correct syntax to filter rows in a DataFrame df where the column 'name' contains the substring 'cat'?

easy

A. df[df['name'].str.contain('cat')]

B. df[df['name'].contains('cat')]

C. df[df['name'].str.contains('cat')]

D. df[df['name'].str.contains(cat)]

Solution

1. Step 1: Identify correct method usage

   The correct method is str.contains called on the Series with the pattern as a string argument.

2. Step 2: Check syntax details

   df[df['name'].str.contains('cat')] uses str.contains('cat') correctly with quotes around 'cat'. df[df['name'].contains('cat')] misses str, C has a typo, D misses quotes around 'cat'.

3. Final Answer:

   df[df['name'].str.contains('cat')] -> Option C

4. Quick Check:

   Use str.contains with string pattern in quotes [OK]

Hint: Use df['col'].str.contains('pattern') with quotes [OK]

Common Mistakes:

• Forgetting to use .str before contains
• Missing quotes around the pattern string
• Typo in method name like contain instead of contains

3. Given the DataFrame df below:

import pandas as pd
df = pd.DataFrame({'text': ['apple pie', 'banana split', 'cherry tart', 'apple turnover']})

What will df[df['text'].str.contains('apple')] return?

medium

A. Rows where 'text' contains 'apple': 'apple pie' and 'apple turnover'.

B. Rows where 'text' contains 'banana' only.

C. All rows because 'apple' is in every string.

D. An empty DataFrame because 'apple' is not found.

Solution

1. Step 1: Check which rows contain 'apple'

   Rows 0 ('apple pie') and 3 ('apple turnover') contain the substring 'apple'.

2. Step 2: Understand filtering result

   The filter returns only these rows, excluding others like 'banana split' and 'cherry tart'.

3. Final Answer:

   Rows where 'text' contains 'apple': 'apple pie' and 'apple turnover'. -> Option A

4. Quick Check:

   Filter keeps rows with 'apple' substring [OK]

Hint: Filter keeps only rows where pattern is found [OK]

Common Mistakes:

• Assuming all rows are returned
• Confusing 'apple' with other fruits
• Expecting a count instead of filtered rows

4. Consider this code snippet:

import pandas as pd
df = pd.DataFrame({'col': ['dog', 'cat', 'bird']})
filtered = df[df['col'].str.contains(cat)]
print(filtered)

What is the error and how to fix it?

medium

A. TypeError because df is not defined; import pandas first.

B. NameError because 'cat' is not quoted; fix by using 'cat' in quotes.

C. AttributeError because str.contains is misspelled; fix spelling.

D. No error; code runs correctly.

Solution

1. Step 1: Identify cause of error

   The pattern 'cat' is used without quotes, so Python treats it as a variable, causing NameError.

2. Step 2: Correct the syntax

   Put 'cat' in quotes: str.contains('cat') to pass a string pattern.

3. Final Answer:

   NameError because 'cat' is not quoted; fix by using 'cat' in quotes. -> Option B

4. Quick Check:

   Pattern must be a string in quotes [OK]

Hint: Always quote string patterns in str.contains [OK]

Common Mistakes:

• Forgetting quotes around pattern
• Assuming pattern can be variable without definition
• Ignoring error messages about undefined names

5. You have a DataFrame df with a column 'comments' containing text. You want to filter rows where 'comments' contain either 'error' or 'fail' (case insensitive). Which code correctly does this?

hard

A. df[df['comments'].str.contains('error|fail', case=false)]

B. df[df['comments'].str.contains('error' or 'fail', case=true)]

C. df[df['comments'].str.contains(['error', 'fail'], case=false)]

D. df[df['comments'].str.contains('error|fail', case=true)]

Solution

1. Step 1: Use regex pattern for multiple words

   To match 'error' or 'fail', use regex 'error|fail' inside str.contains.

2. Step 2: Set case insensitive matching

   Use case=false to ignore case differences.

3. Step 3: Check options

   df[df['comments'].str.contains('error|fail', case=false)] correctly uses regex and case=false. df[df['comments'].str.contains('error' or 'fail', case=true)] misuses 'or' inside string, C passes list instead of string, A uses case=true which is case sensitive.

4. Final Answer:

   df[df['comments'].str.contains('error|fail', case=false)] -> Option A

5. Quick Check:

   Use regex with | and case=false for case-insensitive OR [OK]

Hint: Use regex 'word1|word2' with case=false for OR matching [OK]

Common Mistakes:

• Using Python 'or' inside string instead of regex |
• Passing list instead of string pattern
• Forgetting case=false for case-insensitive search