Text cleaning pipeline in Data Analysis Python - Mini Project: Build & Apply

Text cleaning pipeline

📖 Scenario: You work in a company that collects customer feedback. The feedback text often has extra spaces, uppercase letters, and punctuation that make it hard to analyze. You want to clean this text to prepare it for analysis.

🎯 Goal: Build a simple text cleaning pipeline that removes punctuation, converts text to lowercase, and strips extra spaces from a list of feedback messages.

📋 What You'll Learn

Create a list of feedback messages with exact given texts

Create a variable for punctuation characters to remove

Use a loop to clean each message by removing punctuation, converting to lowercase, and stripping spaces

Print the cleaned list of messages

💡 Why This Matters

🌍 Real World

Cleaning text data is important before analyzing customer feedback, reviews, or social media posts to get accurate insights.

💼 Career

Data analysts and data scientists often clean text data as a first step in text analysis or natural language processing tasks.

1

Create the list of feedback messages

Create a list called feedback with these exact strings: ' Great product! ', 'Needs improvement...', 'Excellent service!!!', ' Will buy again. '

Data Analysis Python

# Create the list called feedback with the exact messages
# Your code here

Hint

Use square brackets to create a list and include the exact strings with quotes.

2

Create the punctuation string

Create a variable called punctuation and set it to the string '!.,' which contains punctuation characters to remove.

Data Analysis Python

feedback = [' Great product!  ', 'Needs improvement...', 'Excellent service!!!', '  Will buy again. ']
# Create the punctuation string
# Your code here

Hint

Assign the exact string '!.,' to the variable punctuation.

3

Clean the feedback messages

Create an empty list called cleaned_feedback. Use a for loop with variable message to go through each item in feedback. Inside the loop, remove all characters in punctuation from message, convert it to lowercase, and strip spaces. Append the cleaned message to cleaned_feedback.

Data Analysis Python

feedback = [' Great product!  ', 'Needs improvement...', 'Excellent service!!!', '  Will buy again. ']
# Create cleaned_feedback list and clean each message
# Your code here

Hint

Use nested loops: outer for each message, inner for each punctuation character to remove. Then lowercase and strip spaces.

4

Print the cleaned feedback list

Write a print statement to display the cleaned_feedback list.

Data Analysis Python

feedback = [' Great product!  ', 'Needs improvement...', 'Excellent service!!!', '  Will buy again. ']
punctuation = '!.,'
cleaned_feedback = []
for message in feedback:
    for p in punctuation:
        message = message.replace(p, '')
    message = message.lower().strip()
    cleaned_feedback.append(message)
# Print the cleaned_feedback list
# Your code here

Hint

Use print(cleaned_feedback) to show the final cleaned list.

Practice

1. What is the main purpose of a text cleaning pipeline in data analysis?

easy

A. To translate text into another language

B. To make text uniform and easier to analyze

C. To increase the length of the text

D. To add more punctuation to the text

Solution

1. Step 1: Understand text cleaning goal

   Text cleaning aims to prepare text data by making it consistent and easier to work with.

2. Step 2: Identify correct purpose

   Adding punctuation or increasing length does not help analysis; uniformity does.

3. Final Answer:

   To make text uniform and easier to analyze -> Option B

4. Quick Check:

   Text cleaning = uniform text [OK]

Hint: Text cleaning makes text consistent for analysis [OK]

Common Mistakes:

• Thinking cleaning adds punctuation
• Confusing cleaning with translation
• Believing cleaning increases text length

2. Which of the following Python code snippets correctly converts a string text to lowercase?

easy

A. text.lower()

B. lower(text)

C. text.toLowerCase()

D. text.lowercase()

Solution

1. Step 1: Recall Python string method for lowercase

   Python strings have a method called lower() to convert text to lowercase.

2. Step 2: Check each option

   text.lower() uses correct syntax. Options B, C, D are invalid in Python.

3. Final Answer:

   text.lower() -> Option A

4. Quick Check:

   Python lowercase method = lower() [OK]

Hint: Use text.lower() to lowercase strings in Python [OK]

Common Mistakes:

• Using JavaScript method toLowerCase() in Python
• Trying a non-existent function lower() outside string
• Using incorrect method name like lowercase()

3. Given the following Python code, what is the output of cleaned_text?

import string
text = " Hello, World!  "
cleaned_text = text.lower().strip().translate(str.maketrans('', '', string.punctuation))
print(cleaned_text)

medium

A. " hello world "

B. "hello, world!"

C. "hello world"

D. "HELLO WORLD"

Solution

1. Step 1: Apply lower() and strip()

   The text becomes "hello, world!" after lowering and removing spaces at ends.

2. Step 2: Remove punctuation with translate()

   All punctuation marks like comma and exclamation are removed, resulting in "hello world".

3. Final Answer:

   "hello world" -> Option C

4. Quick Check:

   Lowercase + strip + remove punctuation = "hello world" [OK]

Hint: Chain methods: lower, strip, then remove punctuation [OK]

Common Mistakes:

• Forgetting to strip spaces
• Not removing punctuation correctly
• Confusing case conversion order

4. Identify the error in this text cleaning function and choose the fix:

def clean_text(text):
    text = text.lower()
    text = text.strip
    return text

print(clean_text("  DATA Science  "))

medium

A. Change text.strip to text.strip()

B. Change text.lower() to lower(text)

C. Add text = text.replace() to remove spaces

D. Remove the return statement

Solution

1. Step 1: Check method usage

   strip is a method and must be called with parentheses ().

2. Step 2: Identify fix

   Changing text.strip to text.strip() correctly removes spaces.

3. Final Answer:

   Change text.strip to text.strip() -> Option A

4. Quick Check:

   Method calls need parentheses [OK]

Hint: Always call string methods with parentheses [OK]

Common Mistakes:

• Forgetting parentheses on methods
• Using wrong syntax for lower()
• Removing return statement by mistake

5. You have a list of sentences with inconsistent spacing and punctuation:

texts = ["Data Science!  ", "  Machine Learning.", "AI, is fun!"]

Which code snippet correctly cleans all texts by making them lowercase, removing punctuation, and trimming spaces?

hard

A. cleaned = [t.strip().lower() for t in texts]

B. cleaned = [t.lower().replace(' ', '') for t in texts]

C. cleaned = [t.translate(str.maketrans('', '', string.punctuation)) for t in texts]

D. import string
cleaned = [t.lower().strip().translate(str.maketrans('', '', string.punctuation)) for t in texts]

Solution

1. Step 1: Understand cleaning steps needed

   We need lowercase, remove punctuation, and trim spaces for each text.

2. Step 2: Check each option

   import string cleaned = [t.lower().strip().translate(str.maketrans('', '', string.punctuation)) for t in texts] applies all three steps correctly using list comprehension. Others miss one or more steps.

3. Final Answer:

   import string cleaned = [t.lower().strip().translate(str.maketrans('', '', string.punctuation)) for t in texts] -> Option D

4. Quick Check:

   All steps in list comprehension = import string cleaned = [t.lower().strip().translate(str.maketrans('', '', string.punctuation)) for t in texts] [OK]

Hint: Use list comprehension chaining all cleaning steps [OK]

Common Mistakes:

• Missing punctuation removal
• Not trimming spaces
• Removing spaces incorrectly with replace