Punctuation and special character removal in NLP - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Output of punctuation removal code

What is the output of the following Python code that removes punctuation and special characters from a text string?

NLP

import re
text = "Hello, world! Welcome to AI & ML." 
clean_text = re.sub(r'[^\w\s]', '', text)
print(clean_text)

A Hello world Welcome to AI ML

B Hello, world! Welcome to AI & ML.

C Hello world! Welcome to AI & ML

D Hello world Welcome to AI & ML.

💡 Hint

Look at the regular expression pattern used in re.sub to remove characters.

The pattern '[^\w\s]' matches any character that is NOT a word character or whitespace. So punctuation and special characters like ',' '!' '&' are removed.

❓ Model Choice

intermediate

Best model for text preprocessing with punctuation removal

Which model type is best suited for handling text data that requires punctuation and special character removal before training?

A Convolutional Neural Network (CNN) for image classification

B Recurrent Neural Network (RNN) for sequence data

C Linear Regression for numerical prediction

D K-Means clustering for unsupervised grouping

💡 Hint

Think about models designed to process sequences of words.

RNNs are designed to handle sequences like text and benefit from preprocessing steps like punctuation removal to improve input quality.

❓ Hyperparameter

advanced

Choosing tokenizer settings for punctuation removal

When using a tokenizer in NLP, which setting helps ensure punctuation and special characters are removed during tokenization?

A Set 'filters' parameter to include punctuation characters

B Set 'strip_accents=True' to remove accents from characters

C Set 'lowercase=True' to convert all text to lowercase

D Set 'max_len' to limit the maximum token length

💡 Hint

Look for the parameter that controls which characters are removed during tokenization.

The 'filters' parameter in tokenizers specifies characters to remove, including punctuation and special characters.

❓ Metrics

advanced

Effect of punctuation removal on text classification accuracy

If you train a text classification model on raw text and then on text with punctuation removed, what is the most likely effect on accuracy?

A Accuracy will always increase because punctuation adds noise

B Accuracy remains exactly the same regardless of punctuation

C Accuracy will always decrease because punctuation carries important meaning

D Accuracy may increase or decrease depending on the dataset and task

💡 Hint

Consider how punctuation might affect different types of text data.

Punctuation can sometimes add useful information or noise, so its removal can help or hurt accuracy depending on context.

🔧 Debug

expert

Debugging punctuation removal code

What error does the following code raise when trying to remove punctuation from text? import string text = "Hello, world!" clean_text = text.translate(str.maketrans('', '', string.punctuation)) print(clean_text)

NLP

import string
text = "Hello, world!"
clean_text = text.translate(str.maketrans('', '', string.punctuation))
print(clean_text)

A TypeError: translate() argument must be a mapping or None

B SyntaxError: invalid syntax in maketrans

C No error, output: Hello world

D AttributeError: 'str' object has no attribute 'translate'

💡 Hint

Check the usage of str.maketrans and translate methods.

The code correctly uses str.maketrans to remove punctuation and prints 'Hello world' without errors.