Text preprocessing pipelines in NLP - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to convert all text to lowercase.

NLP

text = "Hello World!"
processed_text = text.[1]()

Drag options to blanks, or click blank then click option'

Alower

Bupper

Ccapitalize

Dtitle

Attempts:

3 left

💡 Hint

Common Mistakes

Using upper() instead of lower()

Using capitalize() which only changes the first letter

Using lower() converts all characters in the string to lowercase, which is a common first step in text preprocessing.

2fill in blank

medium

Complete the code to split the text into words.

NLP

text = "Machine learning is fun"
words = text.[1]()

Drag options to blanks, or click blank then click option'

Astrip

Bjoin

Creplace

Dsplit

Attempts:

3 left

💡 Hint

Common Mistakes

Using join() which combines words

Using replace() which changes characters

The split() method breaks a string into a list of words using spaces by default.

3fill in blank

hard

Fix the error in the code to remove punctuation from the text.

NLP

import string
text = "Hello, world!"
clean_text = text.translate(str.maketrans('', '', [1]))

Drag options to blanks, or click blank then click option'

Astring.punctuation

Bstring.whitespace

Cstring.ascii_letters

Dstring.digits

Attempts:

3 left

💡 Hint

Common Mistakes

Using whitespace which removes spaces

Using ascii_letters which removes letters

string.punctuation contains all punctuation characters. Removing them cleans the text.

4fill in blank

hard

Fill both blanks to create a list of words without stopwords.

NLP

stopwords = {'is', 'the', 'and'}
words = ['this', 'is', 'fun']
filtered = [word for word in words if word [1] stopwords]

Drag options to blanks, or click blank then click option'

A==

Bin

Cnot in

D!=

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'in' which keeps stopwords

Using '==' which compares equality

Using not in filters out words that are in the stopwords set.

5fill in blank

hard

Fill all three blanks to create a dictionary of word counts from a list.

NLP

words = ['apple', 'banana', 'apple']
word_counts = [1]((word, [2]) for word in words if words.count(word) [3] 1)

Drag options to blanks, or click blank then click option'

Adict

Bwords.count(word)

C>

Dlist

Attempts:

3 left

💡 Hint

Common Mistakes

Using list instead of dict

Using '<' instead of '>' in condition

We use dict to create a dictionary, count each word with words.count(word), and filter words that appear more than once with >.

Practice

1. What is the main purpose of a text preprocessing pipeline in NLP?

easy

A. To train the machine learning model directly

B. To generate new text data automatically

C. To clean and prepare text data step-by-step for models

D. To visualize text data in graphs

Solution

1. Step 1: Understand the role of preprocessing

   Preprocessing cleans and prepares raw text so models can understand it better.

2. Step 2: Identify pipeline benefits

   Pipelines organize these steps neatly and make the process repeatable.

3. Final Answer:

   To clean and prepare text data step-by-step for models -> Option C

4. Quick Check:

   Preprocessing pipeline = clean and prepare text [OK]

Hint: Pipelines organize cleaning steps before modeling [OK]

Common Mistakes:

• Confusing preprocessing with model training
• Thinking pipelines generate new text
• Assuming pipelines visualize data

2. Which of the following is the correct way to chain text preprocessing steps in Python using a pipeline?

easy

A. pipeline = [tokenize, lowercase, remove_stopwords]

B. pipeline = Pipeline(steps=[('tokenize', tokenize), ('lowercase', lowercase), ('stop', remove_stopwords)])

C. pipeline = tokenize + lowercase + remove_stopwords

D. pipeline = tokenize.lowercase.remove_stopwords()

Solution

1. Step 1: Recognize pipeline syntax

   In Python, pipelines are often created using a Pipeline class with named steps.

2. Step 2: Check options

   pipeline = Pipeline(steps=[('tokenize', tokenize), ('lowercase', lowercase), ('stop', remove_stopwords)]) correctly uses Pipeline with steps as tuples of (name, function).

3. Final Answer:

   pipeline = Pipeline(steps=[('tokenize', tokenize), ('lowercase', lowercase), ('stop', remove_stopwords)]) -> Option B

4. Quick Check:

   Pipeline uses steps list with (name, function) tuples [OK]

Hint: Use Pipeline class with named steps list [OK]

Common Mistakes:

• Trying to chain functions with dots or plus signs
• Not naming steps in the pipeline
• Using list of functions without Pipeline wrapper

3. Given the following code snippet, what will be the output of processed_text?

def lowercase(text):
    return text.lower()

def remove_punctuation(text):
    return ''.join(c for c in text if c.isalnum() or c.isspace())

text = "Hello, World!"

pipeline = [lowercase, remove_punctuation]

processed_text = text
for step in pipeline:
    processed_text = step(processed_text)

print(processed_text)

medium

A. hello world

B. Hello World

C. hello, world!

D. HELLO WORLD

Solution

1. Step 1: Apply lowercase function

   "Hello, World!" becomes "hello, world!" after lowercase.

2. Step 2: Apply remove_punctuation function

   Removes commas and exclamation marks, leaving "hello world".

3. Final Answer:

   hello world -> Option A

4. Quick Check:

   Lowercase + remove punctuation = "hello world" [OK]

Hint: Apply steps one by one on text [OK]

Common Mistakes:

• Forgetting to lowercase before removing punctuation
• Assuming punctuation remains
• Confusing case sensitivity

4. Identify the error in this text preprocessing pipeline code and select the fix:

def tokenize(text):
    return text.split()

def remove_stopwords(words):
    stopwords = ['the', 'is', 'at']
    return [w for w in words if w not in stopwords]

text = "The cat is at the door"

pipeline = [tokenize, remove_stopwords]

processed = text
for step in pipeline:
    processed = step(processed)

print(processed)

medium

A. Define stopwords outside the function

B. Add join after remove_stopwords to convert list back to string

C. Replace split() with list() in tokenize

D. Change text to lowercase before tokenizing

Solution

1. Step 1: Analyze stopwords matching

   Stopwords are lowercase but input text has capitalized words, so matching fails.

2. Step 2: Fix by lowercasing text before tokenizing

   Lowercasing ensures stopwords match and are removed correctly.

3. Final Answer:

   Change text to lowercase before tokenizing -> Option D

4. Quick Check:

   Lowercase text first to match stopwords [OK]

Hint: Lowercase text before removing stopwords [OK]

Common Mistakes:

• Ignoring case mismatch in stopwords
• Trying to join list without need
• Changing split() to list() incorrectly

5. You want to build a text preprocessing pipeline that: 1. Converts text to lowercase 2. Removes punctuation 3. Tokenizes text into words 4. Removes stopwords Which of the following pipeline orders is correct to ensure proper processing?

hard

A. Lowercase -> Remove punctuation -> Tokenize -> Remove stopwords

B. Tokenize -> Lowercase -> Remove stopwords -> Remove punctuation

C. Remove stopwords -> Tokenize -> Lowercase -> Remove punctuation

D. Remove punctuation -> Remove stopwords -> Tokenize -> Lowercase

Solution

1. Step 1: Start with lowercase

   Lowercasing first ensures uniform text for all later steps.

2. Step 2: Remove punctuation before tokenizing

   Removing punctuation cleans text so tokens are words only.

3. Step 3: Tokenize then remove stopwords

   Tokenizing splits text into words, then stopwords can be removed from tokens.

4. Final Answer:

   Lowercase -> Remove punctuation -> Tokenize -> Remove stopwords -> Option A

5. Quick Check:

   Correct pipeline order = A [OK]

Hint: Lowercase, clean, tokenize, then filter stopwords [OK]

Common Mistakes:

• Tokenizing before cleaning punctuation
• Removing stopwords before tokenizing
• Not lowercasing first