str.replace() for substitution in Pandas - Step-by-Step Execution

Concept Flow - str.replace() for substitution

Start with DataFrame column

↓

Call str.replace(old, new)

↓

Search each string for 'old'

↓

Replace 'old' with 'new'

↓

Return new Series with replaced strings

The method takes a column of text, finds all occurrences of a substring, replaces them, and returns the updated column.

Execution Sample

Pandas

import pandas as pd
s = pd.Series(['cat', 'dog', 'catdog'])
s_new = s.str.replace('cat', 'mouse')
print(s_new)

Replace 'cat' with 'mouse' in each string of the Series.

Execution Table

Step	Original String	Search for 'cat'	Replace with 'mouse'	Resulting String
1	cat	Found 'cat' at start	Replace 'cat' with 'mouse'	mouse
2	dog	No 'cat' found	No replacement	dog
3	catdog	Found 'cat' at start	Replace 'cat' with 'mouse'	mousedog
4	End	All strings processed	Return new Series	['mouse', 'dog', 'mousedog']

💡 All strings checked and replacements done, method returns new Series.

Variable Tracker

Variable	Start	After 1	After 2	After 3	Final
s	['cat', 'dog', 'catdog']	['cat', 'dog', 'catdog']	['cat', 'dog', 'catdog']	['cat', 'dog', 'catdog']	['cat', 'dog', 'catdog']
s_new	N/A	['mouse', 'dog', 'catdog']	['mouse', 'dog', 'catdog']	['mouse', 'dog', 'mousedog']	['mouse', 'dog', 'mousedog']

Key Moments - 2 Insights

Why does the original Series 's' not change after str.replace()?

Because str.replace() returns a new Series with replacements; it does not modify the original Series in place, as shown in the variable_tracker where 's' stays the same.

What happens if the substring to replace is not found in a string?

No replacement occurs and the original string remains unchanged, as seen in step 2 of the execution_table where 'dog' has no 'cat' and stays 'dog'.

Visual Quiz - 3 Questions

Look at the execution_table, what is the resulting string after step 3?

A mousedog

B catdog

C mouse

D dog

💡 Hint

Check the 'Resulting String' column in row for step 3.

At which step does the method find no substring to replace?

A Step 1

B Step 2

C Step 3

D Step 4

💡 Hint

Look at the 'Search for cat' column to find where 'No cat found' is noted.

If we replaced 'dog' with 'wolf' instead, what would be the final Series after step 4?

A ['cat', 'dog', 'catdog']

B ['mouse', 'dog', 'mousedog']

C ['cat', 'wolf', 'catwolf']

D ['wolf', 'wolf', 'wolf']

💡 Hint

Think about replacing 'dog' with 'wolf' in each string, similar to the example in execution_table.

Concept Snapshot

pandas Series.str.replace(old, new)
- Finds all occurrences of 'old' substring in each string
- Replaces them with 'new'
- Returns a new Series, original unchanged
- Works element-wise on Series of strings
- Useful for quick text substitutions

Full Transcript

This visual trace shows how pandas Series.str.replace() works step-by-step. Starting with a Series of strings, the method searches each string for the substring to replace. If found, it substitutes the new substring and creates a new Series with these changes. The original Series remains unchanged. The execution table tracks each string's replacement process, and the variable tracker shows how the new Series builds up. Key moments clarify why the original data is not modified and what happens when the substring is missing. The quiz tests understanding of the replacement results and behavior.

Practice

1. What does the pandas str.replace() method do when applied to a DataFrame column?

easy

A. It sorts the column values in ascending order.

B. It deletes the entire column from the DataFrame.

C. It converts all strings in the column to uppercase.

D. It replaces specified substrings in each string of the column with a new substring.

Solution

1. Step 1: Understand the purpose of str.replace()

   This method targets each string in a pandas column and looks for a substring to replace.

2. Step 2: Identify the effect on the column

   It creates a new column or series where the specified substring is replaced by the new substring in every string.

3. Final Answer:

   It replaces specified substrings in each string of the column with a new substring. -> Option D

4. Quick Check:

   str.replace() = substring substitution [OK]

Hint: Think: change parts of text in each cell [OK]

Common Mistakes:

• Confusing replace with deleting columns
• Thinking it changes case of strings
• Assuming it sorts data

2. Which of the following is the correct syntax to replace all occurrences of 'cat' with 'dog' in a pandas column named pets?

easy

A. df['pets'].str.replace('cat', 'dog')

B. df['pets'].replace('cat', 'dog')

C. df.str.replace('cat', 'dog')['pets']

D. df['pets'].replace_all('cat', 'dog')

Solution

1. Step 1: Identify the correct method for string replacement

   In pandas, str.replace() is used on a Series of strings to replace substrings.

2. Step 2: Check the syntax for applying str.replace() on a column

   The correct syntax is df['column'].str.replace(old, new). df['pets'].str.replace('cat', 'dog') matches this.

3. Final Answer:

   df['pets'].str.replace('cat', 'dog') -> Option A

4. Quick Check:

   Use .str.replace() on Series for string substitution [OK]

Hint: Use .str.replace() on the column, not .replace() alone [OK]

Common Mistakes:

• Using .replace() directly on the DataFrame column
• Trying to call .str.replace() on the whole DataFrame
• Using non-existent methods like replace_all

3. Given the DataFrame:

import pandas as pd
df = pd.DataFrame({'fruit': ['apple', 'banana', 'grape', 'pineapple']})
df['fruit_new'] = df['fruit'].str.replace('apple', 'orange')

What is the content of df['fruit_new']?

medium

A. ['orange', 'banana', 'grape', 'pineapple']

B. ['apple', 'banana', 'grape', 'pineapple']

C. ['orange', 'banana', 'grape', 'pineorange']

D. ['orange', 'orange', 'orange', 'orange']

Solution

1. Step 1: Understand what str.replace('apple', 'orange') does

   It replaces every occurrence of 'apple' in each string with 'orange'.

2. Step 2: Apply replacement to each value

   'apple' becomes 'orange', 'banana' unchanged, 'grape' unchanged, 'pineapple' contains 'apple' at the end, so it becomes 'pineorange'.

3. Final Answer:

   ['orange', 'banana', 'grape', 'pineorange'] -> Option C

4. Quick Check:

   Substring 'apple' replaced wherever found [OK]

Hint: Replace changes substrings inside words too [OK]

Common Mistakes:

• Assuming only exact matches replaced, not substrings
• Thinking 'pineapple' stays unchanged
• Confusing replacement with full string swap

4. What is wrong with this code snippet?

df['col'].str.replace('a', 'b')

It does not change df['col'] as expected.

medium

A. The result is not assigned back to df['col'] to save changes.

B. The method str.replace() does not exist in pandas.

C. The arguments 'a' and 'b' must be integers, not strings.

D. You must use df.replace() instead of str.replace().

Solution

1. Step 1: Understand that str.replace() returns a new Series

   The method does not change the original column in place; it returns a modified copy.

2. Step 2: Identify the missing assignment

   To keep changes, assign the result back: df['col'] = df['col'].str.replace('a', 'b').

3. Final Answer:

   The result is not assigned back to df['col'] to save changes. -> Option A

4. Quick Check:

   Assign result to keep changes [OK]

Hint: Always assign back to save string replacements [OK]

Common Mistakes:

• Expecting in-place change without assignment
• Confusing str.replace() with DataFrame.replace()
• Thinking str.replace() modifies original data silently

5. You have a DataFrame with a column emails containing email addresses. You want to anonymize the domain part by replacing everything after '@' with 'example.com'. Which code correctly does this?

hard

A. df['emails'].replace(r'@.*', '@example.com')

B. df['emails'].str.replace(r'@.*', '@example.com', regex=True)

C. df['emails'].str.replace('example.com', '@')

D. df['emails'].str.replace('@', '@example.com')

Solution

1. Step 1: Identify the pattern to replace

   The domain part starts with '@' and continues to the end, so the regex '@.*' matches it.

2. Step 2: Use str.replace() with regex=True to replace domain

   df['emails'].str.replace(r'@.*', '@example.com', regex=True) uses regex to replace everything after '@' with '@example.com'.

3. Step 3: Check other options

   A uses Series.replace() which defaults to regex=False, treating '@.*' literally. B replaces only '@' leaving the domain. C replaces in the wrong direction.

4. Final Answer:

   df['emails'].str.replace(r'@.*', '@example.com', regex=True) -> Option B

5. Quick Check:

   Use regex in str.replace() for domain substitution [OK]

Hint: Use regex '@.*' with regex=True to replace domain [OK]

Common Mistakes:

• Not using regex=True for pattern matching
• Replacing only '@' without domain
• Using DataFrame.replace instead of str.replace