Extracting with str.extract (regex) in Data Analysis Python - Cheat Sheet & Quick Revision

Recall & Review

beginner

What does the str.extract() method do in pandas?

It extracts groups from strings in a pandas Series or DataFrame column using a regular expression (regex). It returns the matched groups as new columns.

beginner

How do you specify which part of the string to extract using str.extract()?

You use parentheses () in the regex pattern to define capture groups. Only the text matched inside these groups is extracted.

intermediate

What type of object does str.extract() return when extracting one group vs multiple groups?

If one group is extracted, it returns a Series. If multiple groups are extracted, it returns a DataFrame with one column per group.

beginner

Why is it useful to use str.extract() with regex in data cleaning?

It helps pull out specific parts of messy text data, like extracting dates, codes, or names, making the data easier to analyze.

intermediate

Example: What does df['col'].str.extract(r'(\d{3})-(\d{2})') extract?

It extracts two groups of digits separated by a dash: the first group with exactly 3 digits, and the second group with exactly 2 digits, returning them as two columns.

What symbol in regex defines a capture group for str.extract()?

A ()

B []

C {}

D <>

Parentheses () define capture groups in regex, which str.extract() uses to extract parts of strings.

If your regex has two capture groups, what does str.extract() return?

A A Series with one column

B A DataFrame with two columns

C A list of strings

D A single string

Two capture groups produce a DataFrame with one column per group.

Which pandas object can you use str.extract() on?

A DataFrame

B Series

C Both DataFrame and Series

D null

str.extract() is used on pandas Series (single columns). For DataFrames, you apply it on a column (Series).

What happens if the regex pattern does not match any part of the string?

A Returns the original string

B Returns NaN for that row

C Raises an error

D Returns an empty string

If no match is found, str.extract() returns NaN for that row.

Which of these regex patterns extracts a 4-digit year from a string?

A r'(\d{4})'

B r'\d{2}'

C r'(\w{4})'

D r'\d{5}'

The pattern r'(\d{4})' captures exactly 4 digits, suitable for a year.

Explain how you would use str.extract() to pull out an area code from phone numbers in a pandas Series.

Think about the pattern for area codes like three digits inside parentheses or at the start.

Describe the difference in output when extracting one group versus multiple groups with str.extract().

Consider how pandas organizes extracted data based on number of groups.

Practice

1. What does the str.extract method do in pandas when used with a regex pattern?

easy

A. It replaces matching text with a new string.

B. It deletes rows that don't match the pattern.

C. It pulls out matching parts of text into a new DataFrame.

D. It counts how many times the pattern appears.

Solution

1. Step 1: Understand str.extract purpose

   The method is designed to find parts of text that match a regex pattern and extract them.

2. Step 2: Check what it returns

   It returns a new DataFrame with the extracted groups or NaN if no match is found.

3. Final Answer:

   It pulls out matching parts of text into a new DataFrame. -> Option C

4. Quick Check:

   str.extract extracts text parts [OK]

Hint: Remember: extract pulls out text pieces using patterns [OK]

Common Mistakes:

• Thinking it deletes rows instead of extracting text
• Confusing extract with replace or count methods
• Expecting a single string instead of a DataFrame

2. Which of the following is the correct syntax to extract a 3-digit number from a pandas Series df['code'] using str.extract?

easy

A. df['code'].str.extract('\d{3}')

B. df['code'].str.extract(r'(\d{3})')

C. df['code'].str.extract(r'\d{3}')

D. df['code'].str.extract('(\d{3})')

Solution

1. Step 1: Identify correct regex string format

   Regex pattern must be a raw string (r'...') to handle backslashes properly.

2. Step 2: Check parentheses for capturing group

   Parentheses are needed to capture the group to extract.

3. Final Answer:

   df['code'].str.extract(r'(\d{3})') -> Option B

4. Quick Check:

   Use raw string with parentheses for capture [OK]

Hint: Use raw string and parentheses for capture groups [OK]

Common Mistakes:

• Missing parentheses around the pattern
• Not using raw string causing escape errors
• Using double backslashes incorrectly

3. Given the following code:

import pandas as pd
df = pd.DataFrame({'text': ['ID123', 'ID456', 'NoID', 'ID789']})
df['number'] = df['text'].str.extract(r'ID(\d{3})')

What will be the content of the new column number?

medium

A. [NaN, NaN, NaN, NaN]

B. ['ID123', 'ID456', 'NoID', 'ID789']

C. ['123', '456', 'NoID', '789']

D. ['123', '456', NaN, '789']

Solution

1. Step 1: Understand the regex pattern

   The pattern 'ID(\d{3})' looks for 'ID' followed by exactly 3 digits and extracts the digits.

2. Step 2: Apply extraction to each row

   'ID123' -> '123', 'ID456' -> '456', 'NoID' -> no match so NaN, 'ID789' -> '789'.

3. Final Answer:

   ['123', '456', NaN, '789'] -> Option D

4. Quick Check:

   Extract digits after 'ID' or NaN if no match [OK]

Hint: Extracted groups appear; no match gives NaN [OK]

Common Mistakes:

• Expecting full string instead of just digits
• Not realizing unmatched rows become NaN
• Confusing extracted text with original

4. Consider this code snippet:

import pandas as pd
df = pd.DataFrame({'info': ['abc123', 'def456', 'ghi789']})
df['extracted'] = df['info'].str.extract('\d{3}')

What is the issue with this code?

medium

A. The regex pattern lacks parentheses for capturing groups.

B. The regex pattern is invalid syntax.

C. The DataFrame column name is incorrect.

D. str.extract cannot extract digits.

Solution

1. Step 1: Check regex pattern for capture groups

   str.extract requires parentheses to capture groups; '\d{3}' has none.

2. Step 2: Understand effect of missing parentheses

   Without parentheses, str.extract returns NaN for all rows because no group is captured.

3. Final Answer:

   The regex pattern lacks parentheses for capturing groups. -> Option A

4. Quick Check:

   Parentheses needed to capture groups [OK]

Hint: Always use parentheses to capture groups in str.extract [OK]

Common Mistakes:

• Forgetting parentheses around regex group
• Assuming str.extract works without capture groups
• Confusing str.extract with str.contains

5. You have a DataFrame df with a column 'data' containing strings like 'Order: #12345, Date: 2023-05-10'. You want to extract the order number and date into separate columns using str.extract. Which regex pattern correctly extracts both parts?

hard

A. r'Order: #(\d+), Date: (\d{4}-\d{2}-\d{2})'

B. r'Order: #(\d+), Date: \d{4}-\d{2}-\d{2}'

C. r'Order: #\d+, Date: (\d{4}-\d{2}-\d{2})'

D. r'Order: #(\d+), Date: (\d{4}/\d{2}/\d{2})'

Solution

1. Step 1: Identify needed capture groups

   We need to capture order number digits and date in 'YYYY-MM-DD' format, so two groups are required.

2. Step 2: Check regex pattern correctness

   r'Order: #(\d+), Date: (\d{4}-\d{2}-\d{2})' has two groups: '(\d+)' for order number and '(\d{4}-\d{2}-\d{2})' for date, matching the format exactly.

3. Final Answer:

   r'Order: #(\d+), Date: (\d{4}-\d{2}-\d{2})' -> Option A

4. Quick Check:

   Two capture groups matching order and date format [OK]

Hint: Use parentheses for each part you want to extract [OK]

Common Mistakes:

• Missing parentheses around one or both groups
• Wrong date format with slashes instead of dashes
• Not capturing order number digits