Extracting with str.extract (regex) in Data Analysis Python - Time & Space Complexity

Time Complexity: Extracting with str.extract (regex)

O(n)

Understanding Time Complexity

We want to understand how the time needed to extract text using regex grows as the data size increases.

How does the extraction time change when we have more rows to process?

Scenario Under Consideration

Analyze the time complexity of the following code snippet.

import pandas as pd

data = pd.Series(['abc123', 'def456', 'ghi789'] * 1000)
pattern = r'(\d+)'
extracted = data.str.extract(pattern)

This code extracts the number part from each string in a pandas Series using a regex pattern.

Identify Repeating Operations

Identify the loops, recursion, array traversals that repeat.

• Primary operation: Applying the regex extraction on each string in the Series.
• How many times: Once per element in the Series, so as many times as the number of rows.

How Execution Grows With Input

As the number of rows grows, the total work grows roughly in direct proportion.

Input Size (n)	Approx. Operations
10	10 regex extractions
100	100 regex extractions
1000	1000 regex extractions

Pattern observation: Doubling the input roughly doubles the work because each string is processed once.

Final Time Complexity

Time Complexity: O(n)

This means the time to extract grows linearly with the number of strings you process.

Common Mistake

[X] Wrong: "Using regex extraction is instant no matter how many rows there are."

[OK] Correct: Each row requires running the regex, so more rows mean more work and more time.

Interview Connect

Understanding how regex extraction scales helps you explain performance when working with text data in real projects.

Self-Check

"What if the regex pattern was more complex and slower to match? How would that affect the time complexity?"

Practice

1. What does the str.extract method do in pandas when used with a regex pattern?

easy

A. It replaces matching text with a new string.

B. It deletes rows that don't match the pattern.

C. It pulls out matching parts of text into a new DataFrame.

D. It counts how many times the pattern appears.

Solution

1. Step 1: Understand str.extract purpose

   The method is designed to find parts of text that match a regex pattern and extract them.

2. Step 2: Check what it returns

   It returns a new DataFrame with the extracted groups or NaN if no match is found.

3. Final Answer:

   It pulls out matching parts of text into a new DataFrame. -> Option C

4. Quick Check:

   str.extract extracts text parts [OK]

Hint: Remember: extract pulls out text pieces using patterns [OK]

Common Mistakes:

• Thinking it deletes rows instead of extracting text
• Confusing extract with replace or count methods
• Expecting a single string instead of a DataFrame

2. Which of the following is the correct syntax to extract a 3-digit number from a pandas Series df['code'] using str.extract?

easy

A. df['code'].str.extract('\d{3}')

B. df['code'].str.extract(r'(\d{3})')

C. df['code'].str.extract(r'\d{3}')

D. df['code'].str.extract('(\d{3})')

Solution

1. Step 1: Identify correct regex string format

   Regex pattern must be a raw string (r'...') to handle backslashes properly.

2. Step 2: Check parentheses for capturing group

   Parentheses are needed to capture the group to extract.

3. Final Answer:

   df['code'].str.extract(r'(\d{3})') -> Option B

4. Quick Check:

   Use raw string with parentheses for capture [OK]

Hint: Use raw string and parentheses for capture groups [OK]

Common Mistakes:

• Missing parentheses around the pattern
• Not using raw string causing escape errors
• Using double backslashes incorrectly

3. Given the following code:

import pandas as pd
df = pd.DataFrame({'text': ['ID123', 'ID456', 'NoID', 'ID789']})
df['number'] = df['text'].str.extract(r'ID(\d{3})')

What will be the content of the new column number?

medium

A. [NaN, NaN, NaN, NaN]

B. ['ID123', 'ID456', 'NoID', 'ID789']

C. ['123', '456', 'NoID', '789']

D. ['123', '456', NaN, '789']

Solution

1. Step 1: Understand the regex pattern

   The pattern 'ID(\d{3})' looks for 'ID' followed by exactly 3 digits and extracts the digits.

2. Step 2: Apply extraction to each row

   'ID123' -> '123', 'ID456' -> '456', 'NoID' -> no match so NaN, 'ID789' -> '789'.

3. Final Answer:

   ['123', '456', NaN, '789'] -> Option D

4. Quick Check:

   Extract digits after 'ID' or NaN if no match [OK]

Hint: Extracted groups appear; no match gives NaN [OK]

Common Mistakes:

• Expecting full string instead of just digits
• Not realizing unmatched rows become NaN
• Confusing extracted text with original

4. Consider this code snippet:

import pandas as pd
df = pd.DataFrame({'info': ['abc123', 'def456', 'ghi789']})
df['extracted'] = df['info'].str.extract('\d{3}')

What is the issue with this code?

medium

A. The regex pattern lacks parentheses for capturing groups.

B. The regex pattern is invalid syntax.

C. The DataFrame column name is incorrect.

D. str.extract cannot extract digits.

Solution

1. Step 1: Check regex pattern for capture groups

   str.extract requires parentheses to capture groups; '\d{3}' has none.

2. Step 2: Understand effect of missing parentheses

   Without parentheses, str.extract returns NaN for all rows because no group is captured.

3. Final Answer:

   The regex pattern lacks parentheses for capturing groups. -> Option A

4. Quick Check:

   Parentheses needed to capture groups [OK]

Hint: Always use parentheses to capture groups in str.extract [OK]

Common Mistakes:

• Forgetting parentheses around regex group
• Assuming str.extract works without capture groups
• Confusing str.extract with str.contains

5. You have a DataFrame df with a column 'data' containing strings like 'Order: #12345, Date: 2023-05-10'. You want to extract the order number and date into separate columns using str.extract. Which regex pattern correctly extracts both parts?

hard

A. r'Order: #(\d+), Date: (\d{4}-\d{2}-\d{2})'

B. r'Order: #(\d+), Date: \d{4}-\d{2}-\d{2}'

C. r'Order: #\d+, Date: (\d{4}-\d{2}-\d{2})'

D. r'Order: #(\d+), Date: (\d{4}/\d{2}/\d{2})'

Solution

1. Step 1: Identify needed capture groups

   We need to capture order number digits and date in 'YYYY-MM-DD' format, so two groups are required.

2. Step 2: Check regex pattern correctness

   r'Order: #(\d+), Date: (\d{4}-\d{2}-\d{2})' has two groups: '(\d+)' for order number and '(\d{4}-\d{2}-\d{2})' for date, matching the format exactly.

3. Final Answer:

   r'Order: #(\d+), Date: (\d{4}-\d{2}-\d{2})' -> Option A

4. Quick Check:

   Two capture groups matching order and date format [OK]

Hint: Use parentheses for each part you want to extract [OK]

Common Mistakes:

• Missing parentheses around one or both groups
• Wrong date format with slashes instead of dashes
• Not capturing order number digits