Why string operations matter in Pandas - Challenge Your Understanding

Challenge - 5 Problems

❓ Predict Output

intermediate

Output of pandas string split and expand

What is the output of this code snippet that splits a column of full names into first and last names?

Pandas

import pandas as pd

df = pd.DataFrame({'Name': ['Alice Smith', 'Bob Jones', 'Carol Lee']})
df[['First', 'Last']] = df['Name'].str.split(' ', expand=True)
df

A

       Name  First   Last
0  Alice Smith  Alice  Smith
1    Bob Jones    Bob  Jones
2   Carol Lee  Carol   Lee

B

       Name  First
0  Alice Smith  Alice
1    Bob Jones    Bob
2   Carol Lee  Carol

C [['Alice', 'Smith'], ['Bob', 'Jones'], ['Carol', 'Lee']]

D

       Name
0  Alice Smith
1    Bob Jones
2   Carol Lee

💡 Hint

Look at how the split with expand=True creates new columns in the DataFrame.

The str.split with expand=True splits the string into separate columns. Here, 'Name' is split into 'First' and 'Last' columns.

❓ data_output

intermediate

Count of rows containing a substring

Given this DataFrame, how many rows contain the substring 'cat' in the 'Animal' column?

Pandas

import pandas as pd

df = pd.DataFrame({'Animal': ['cat', 'dog', 'caterpillar', 'bird', 'scatter']})
count = df['Animal'].str.contains('cat').sum()
count

A 3

B 2

C 4

D 1

💡 Hint

Check which strings have 'cat' anywhere inside them.

The strings 'cat', 'caterpillar', and 'scatter' all contain 'cat', so the count is 3.

🔧 Debug

advanced

Identify the error in string replacement

What error does this code raise when trying to replace 'dog' with 'cat' in the 'Animal' column?

Pandas

import pandas as pd

df = pd.DataFrame({'Animal': ['dog', 'dogfish', 'hotdog']})
df['Animal'].str.replace('dog')

A ValueError: pattern not found

B TypeError: replace() missing 1 required positional argument: 'repl'

C AttributeError: 'Series' object has no attribute 'str'

D No error, returns replaced strings

💡 Hint

Check the required arguments for str.replace method.

The str.replace method requires two arguments: the pattern to replace and the replacement string. Here only one argument is given, causing a TypeError.

❓ visualization

advanced

Visualizing string length distribution

Which option produces a histogram of the lengths of strings in the 'Words' column?

Pandas

import pandas as pd
import matplotlib.pyplot as plt

df = pd.DataFrame({'Words': ['apple', 'banana', 'pear', 'kiwi', 'grape']})

lengths = df['Words'].str.len()
plt.hist(lengths)
plt.xlabel('Length of word')
plt.ylabel('Frequency')
plt.title('Histogram of word lengths')
plt.show()

A A line plot of word lengths over index

B A bar chart of word counts

C A scatter plot of word lengths vs words

D A histogram showing word lengths with bars at lengths 4, 5, 6

💡 Hint

Histogram groups numeric values into bins and shows frequency.

The code calculates string lengths and plots a histogram showing how many words have each length.

🧠 Conceptual

expert

Why string operations are crucial in data science

Which statement best explains why string operations matter in data science?

A String operations are mainly used to speed up numerical calculations in datasets.

B String operations are only useful for formatting output and have little impact on data analysis.

C String operations allow cleaning and transforming text data, which is often messy and unstructured, enabling better analysis.

D String operations replace the need for statistical methods in data science.

💡 Hint

Think about the role of text data in real-world datasets.

Text data is common and often messy. String operations help clean, extract, and transform this data for meaningful analysis.

Practice

1. Why do we use string operations like .str.lower() on pandas columns?

easy

A. To convert numbers into strings

B. To sort the rows in the DataFrame

C. To remove missing values from the column

D. To make text data consistent by changing all letters to lowercase

Solution

1. Step 1: Understand string consistency

   Text data can have mixed uppercase and lowercase letters, which can cause mismatches when comparing or grouping.

2. Step 2: Use .str.lower() to unify text

   Applying .str.lower() converts all letters to lowercase, making data consistent and easier to analyze.

3. Final Answer:

   To make text data consistent by changing all letters to lowercase -> Option D

4. Quick Check:

   String case normalization = consistent text [OK]

Hint: Use .str.lower() to unify text case quickly [OK]

Common Mistakes:

• Thinking .str.lower() changes numbers
• Confusing string operations with sorting
• Assuming it removes missing values

2. Which of the following is the correct syntax to check if a pandas column 'Name' contains the substring 'john' (case insensitive)?

easy

A. df['Name'].str.find('john', case_sensitive=False)

B. df['Name'].contains('john', ignore_case=True)

C. df['Name'].str.contains('john', case=False)

D. df['Name'].str.match('john', ignore_case=True)

Solution

1. Step 1: Identify correct method for substring search

   The pandas string method to check substring presence is .str.contains().

2. Step 2: Use correct parameters for case insensitivity

   The parameter to ignore case is case=False inside .str.contains().

3. Final Answer:

   df['Name'].str.contains('john', case=False) -> Option C

4. Quick Check:

   Substring search with case=False = correct syntax [OK]

Hint: Use .str.contains('text', case=False) for case-insensitive search [OK]

Common Mistakes:

• Using .contains() instead of .str.contains()
• Wrong parameter names like ignore_case
• Using .str.match() which matches whole string

3. What will be the output of this code?

import pandas as pd
df = pd.DataFrame({'City': ['New York', 'los angeles', 'Chicago', 'new orleans']})
df['City_lower'] = df['City'].str.lower()
print(df['City_lower'])

medium

A. 0 new york
1 los angeles
2 chicago
3 new orleans
Name: City_lower, dtype: object

B. 0 New York
1 Los Angeles
2 Chicago
3 New Orleans
Name: City_lower, dtype: object

C. 0 NEW YORK
1 LOS ANGELES
2 CHICAGO
3 NEW ORLEANS
Name: City_lower, dtype: object

D. Error because .str.lower() cannot be applied to DataFrame columns

Solution

1. Step 1: Understand .str.lower() effect on strings

   The method converts all characters in each string to lowercase.

2. Step 2: Apply .str.lower() on 'City' column

   Each city name is converted to lowercase, so 'New York' becomes 'new york', etc.

3. Final Answer:

   0 new york 1 los angeles 2 chicago 3 new orleans Name: City_lower, dtype: object -> Option A

4. Quick Check:

   .str.lower() converts text to lowercase [OK]

Hint: Remember .str.lower() changes all letters to lowercase [OK]

Common Mistakes:

• Expecting original case to remain
• Thinking it converts to uppercase
• Assuming .str.lower() causes error on columns

4. The following code throws an error. What is the mistake?

df = pd.DataFrame({'Name': ['Alice', 'Bob', 'Charlie']})
df['Name_lower'] = df['Name'].lower()

medium

A. You must use .str.lower() to apply string method on a pandas Series

B. The column 'Name' does not exist in the DataFrame

C. The method .lower() should be called without parentheses

D. You cannot create new columns in pandas DataFrame

Solution

1. Step 1: Identify method usage on pandas Series

   Directly calling .lower() on a pandas Series causes error because it's a string method for single strings, not Series.

2. Step 2: Use .str accessor for string methods on Series

   To apply string methods on a whole column, use .str.lower() instead of .lower().

3. Final Answer:

   You must use .str.lower() to apply string method on a pandas Series -> Option A

4. Quick Check:

   Use .str.lower() on Series, not .lower() [OK]

Hint: Use .str before string methods on pandas columns [OK]

Common Mistakes:

• Calling .lower() directly on Series
• Forgetting .str accessor
• Thinking .lower() works on whole column

5. You have a DataFrame with a column 'Email' containing email addresses with mixed cases and extra spaces. Which sequence of string operations will clean the data by trimming spaces and converting all emails to lowercase?

hard

A. df['Email'].lower().strip()

B. df['Email'].str.strip().str.lower()

C. df['Email'].strip().lower()

D. df['Email'].str.lower().strip()

Solution

1. Step 1: Remove extra spaces using .str.strip()

   Use .str.strip() to remove spaces from start and end of each string in the column.

2. Step 2: Convert all text to lowercase with .str.lower()

   Apply .str.lower() after stripping to unify case.

3. Step 3: Chain methods correctly

   Chaining .str.strip().str.lower() applies both operations in order on the Series.

4. Final Answer:

   df['Email'].str.strip().str.lower() -> Option B

5. Quick Check:

   Chain .str.strip() then .str.lower() for clean emails [OK]

Hint: Chain .str.strip() then .str.lower() to clean text [OK]

Common Mistakes:

• Calling .lower() or .strip() without .str
• Wrong order of operations
• Using methods directly on Series without .str