str.replace() for substitution in Pandas - Time & Space Complexity

Time Complexity: str.replace() for substitution

O(n * m)

Understanding Time Complexity

We want to understand how the time it takes to replace text in a pandas column changes as the data grows.

How does the work grow when we replace strings in many rows?

Scenario Under Consideration

Analyze the time complexity of the following code snippet.

import pandas as pd

df = pd.DataFrame({
    'text': ['apple pie', 'banana split', 'apple tart', 'banana bread'] * 1000
})

# Replace 'apple' with 'orange' in the 'text' column
result = df['text'].str.replace('apple', 'orange', regex=False)

This code replaces the word 'apple' with 'orange' in every string of the 'text' column.

Identify Repeating Operations

• Primary operation: Checking and replacing the substring in each string of the column.
• How many times: Once for each row in the DataFrame (n times).

How Execution Grows With Input

As the number of rows grows, the total work grows roughly in direct proportion.

Input Size (n)	Approx. Operations
10	About 10 string checks and replacements
100	About 100 string checks and replacements
1000	About 1000 string checks and replacements

Pattern observation: Doubling the rows roughly doubles the work.

Final Time Complexity

Time Complexity: O(n * m)

This means the time grows with the number of rows (n) and the length of each string (m) because each string is checked and replaced.

Common Mistake

[X] Wrong: "Replacing text in a column is always very fast and does not depend on data size."

[OK] Correct: The operation must look at each string, so more rows or longer strings mean more work and more time.

Interview Connect

Understanding how string operations scale helps you write efficient data cleaning code and explain your choices clearly in interviews.

Self-Check

"What if we replaced a regex pattern instead of a fixed string? How would the time complexity change?"

Practice

1. What does the pandas str.replace() method do when applied to a DataFrame column?

easy

A. It sorts the column values in ascending order.

B. It deletes the entire column from the DataFrame.

C. It converts all strings in the column to uppercase.

D. It replaces specified substrings in each string of the column with a new substring.

Solution

1. Step 1: Understand the purpose of str.replace()

   This method targets each string in a pandas column and looks for a substring to replace.

2. Step 2: Identify the effect on the column

   It creates a new column or series where the specified substring is replaced by the new substring in every string.

3. Final Answer:

   It replaces specified substrings in each string of the column with a new substring. -> Option D

4. Quick Check:

   str.replace() = substring substitution [OK]

Hint: Think: change parts of text in each cell [OK]

Common Mistakes:

• Confusing replace with deleting columns
• Thinking it changes case of strings
• Assuming it sorts data

2. Which of the following is the correct syntax to replace all occurrences of 'cat' with 'dog' in a pandas column named pets?

easy

A. df['pets'].str.replace('cat', 'dog')

B. df['pets'].replace('cat', 'dog')

C. df.str.replace('cat', 'dog')['pets']

D. df['pets'].replace_all('cat', 'dog')

Solution

1. Step 1: Identify the correct method for string replacement

   In pandas, str.replace() is used on a Series of strings to replace substrings.

2. Step 2: Check the syntax for applying str.replace() on a column

   The correct syntax is df['column'].str.replace(old, new). df['pets'].str.replace('cat', 'dog') matches this.

3. Final Answer:

   df['pets'].str.replace('cat', 'dog') -> Option A

4. Quick Check:

   Use .str.replace() on Series for string substitution [OK]

Hint: Use .str.replace() on the column, not .replace() alone [OK]

Common Mistakes:

• Using .replace() directly on the DataFrame column
• Trying to call .str.replace() on the whole DataFrame
• Using non-existent methods like replace_all

3. Given the DataFrame:

import pandas as pd
df = pd.DataFrame({'fruit': ['apple', 'banana', 'grape', 'pineapple']})
df['fruit_new'] = df['fruit'].str.replace('apple', 'orange')

What is the content of df['fruit_new']?

medium

A. ['orange', 'banana', 'grape', 'pineapple']

B. ['apple', 'banana', 'grape', 'pineapple']

C. ['orange', 'banana', 'grape', 'pineorange']

D. ['orange', 'orange', 'orange', 'orange']

Solution

1. Step 1: Understand what str.replace('apple', 'orange') does

   It replaces every occurrence of 'apple' in each string with 'orange'.

2. Step 2: Apply replacement to each value

   'apple' becomes 'orange', 'banana' unchanged, 'grape' unchanged, 'pineapple' contains 'apple' at the end, so it becomes 'pineorange'.

3. Final Answer:

   ['orange', 'banana', 'grape', 'pineorange'] -> Option C

4. Quick Check:

   Substring 'apple' replaced wherever found [OK]

Hint: Replace changes substrings inside words too [OK]

Common Mistakes:

• Assuming only exact matches replaced, not substrings
• Thinking 'pineapple' stays unchanged
• Confusing replacement with full string swap

4. What is wrong with this code snippet?

df['col'].str.replace('a', 'b')

It does not change df['col'] as expected.

medium

A. The result is not assigned back to df['col'] to save changes.

B. The method str.replace() does not exist in pandas.

C. The arguments 'a' and 'b' must be integers, not strings.

D. You must use df.replace() instead of str.replace().

Solution

1. Step 1: Understand that str.replace() returns a new Series

   The method does not change the original column in place; it returns a modified copy.

2. Step 2: Identify the missing assignment

   To keep changes, assign the result back: df['col'] = df['col'].str.replace('a', 'b').

3. Final Answer:

   The result is not assigned back to df['col'] to save changes. -> Option A

4. Quick Check:

   Assign result to keep changes [OK]

Hint: Always assign back to save string replacements [OK]

Common Mistakes:

• Expecting in-place change without assignment
• Confusing str.replace() with DataFrame.replace()
• Thinking str.replace() modifies original data silently

5. You have a DataFrame with a column emails containing email addresses. You want to anonymize the domain part by replacing everything after '@' with 'example.com'. Which code correctly does this?

hard

A. df['emails'].replace(r'@.*', '@example.com')

B. df['emails'].str.replace(r'@.*', '@example.com', regex=True)

C. df['emails'].str.replace('example.com', '@')

D. df['emails'].str.replace('@', '@example.com')

Solution

1. Step 1: Identify the pattern to replace

   The domain part starts with '@' and continues to the end, so the regex '@.*' matches it.

2. Step 2: Use str.replace() with regex=True to replace domain

   df['emails'].str.replace(r'@.*', '@example.com', regex=True) uses regex to replace everything after '@' with '@example.com'.

3. Step 3: Check other options

   A uses Series.replace() which defaults to regex=False, treating '@.*' literally. B replaces only '@' leaving the domain. C replaces in the wrong direction.

4. Final Answer:

   df['emails'].str.replace(r'@.*', '@example.com', regex=True) -> Option B

5. Quick Check:

   Use regex in str.replace() for domain substitution [OK]

Hint: Use regex '@.*' with regex=True to replace domain [OK]

Common Mistakes:

• Not using regex=True for pattern matching
• Replacing only '@' without domain
• Using DataFrame.replace instead of str.replace