Regex operations in Pandas - Step-by-Step Execution

Concept Flow - Regex operations in Pandas

Start with DataFrame

↓

Choose column with text

↓

Apply regex operation

↓

Match/Extract/Replace results

↓

Store or display output

↓

End

Start with a DataFrame, pick a text column, apply regex to find or change patterns, then get the results.

Execution Sample

Pandas

import pandas as pd

df = pd.DataFrame({'text': ['apple123', 'banana456', 'cherry789']})
df['digits'] = df['text'].str.extract('(\d+)')
print(df)

Extract digits from text column using regex and add as new column.

Execution Table

Step	Action	Input	Regex Pattern	Result	Output DataFrame
1	Create DataFrame	{'text': ['apple123', 'banana456', 'cherry789']}		DataFrame with 3 rows	text 0 apple123 1 banana456 2 cherry789
2	Apply str.extract	text column	(\d+)	Extract digits from each string	text digits 0 apple123 123 1 banana456 456 2 cherry789 789
3	Print DataFrame			Show DataFrame with new digits column	text digits 0 apple123 123 1 banana456 456 2 cherry789 789
4	End			Process complete	Final DataFrame shown

💡 All rows processed, digits extracted from text column using regex.

Variable Tracker

Variable	Start	After Step 1	After Step 2	Final
df	undefined	DataFrame with 'text' column	Added 'digits' column with extracted numbers	DataFrame with 'text' and 'digits' columns

Key Moments - 2 Insights

Why do we use double backslashes in the regex pattern '(\\d+)'?

In the execution_table step 2, the pattern '(\\d+)' uses double backslashes because Python strings treat '\\' as a single backslash. This is needed to pass the correct regex '\d+' to pandas.

What happens if the regex does not find a match in a string?

If no match is found, pandas inserts NaN in the output column for that row, as shown in the execution_table step 2 where each row matched digits, but if not, NaN would appear.

Visual Quiz - 3 Questions

Look at the execution_table step 2, what is the value in the 'digits' column for the second row?

A 456

B 789

C 123

D NaN

💡 Hint

Check the 'Result' and 'Output DataFrame' columns in step 2 for the second row.

At which step is the new 'digits' column added to the DataFrame?

A Step 1

B Step 3

C Step 2

D Step 4

💡 Hint

Look at the 'Action' and 'Output DataFrame' columns to see when 'digits' appears.

If the regex pattern was changed to '(\\D+)', what would the 'digits' column contain?

A Only digits extracted

B Only non-digit characters extracted

C Empty strings

D Original text unchanged

💡 Hint

Recall that '\\D' matches non-digit characters, opposite of '\\d'.

Concept Snapshot

Regex operations in Pandas:
- Use df['col'].str methods with regex patterns
- Common methods: extract(), contains(), replace()
- Patterns need double backslashes in Python strings
- Output can be new columns or filtered data
- Missing matches result in NaN values

Full Transcript

This visual trace shows how to use regex operations in pandas. We start with a DataFrame containing text data. We pick a column and apply a regex pattern using str.extract to find digits. The regex pattern '(\\d+)' looks for one or more digits. The extracted digits are added as a new column. Each step updates the DataFrame, and the final output shows the original text and extracted digits side by side. Key points include using double backslashes in regex patterns and handling missing matches with NaN. This step-by-step helps beginners see how regex works inside pandas.