Lowercasing and normalization in NLP - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is lowercasing in text preprocessing?

Lowercasing means converting all letters in text to lowercase. It helps treat words like 'Apple' and 'apple' as the same word.

beginner

Why do we normalize text in NLP?

Normalization makes text consistent by fixing variations like accents, punctuation, or spacing. This helps models understand text better.

intermediate

Give an example of text normalization besides lowercasing.

Removing accents (e.g., changing 'café' to 'cafe') or replacing multiple spaces with a single space are examples of normalization.

intermediate

How does lowercasing affect model vocabulary size?

Lowercasing reduces vocabulary size by merging words that differ only in case, making the model simpler and faster.

advanced

What is a potential downside of lowercasing?

Lowercasing can lose information, like proper nouns or acronyms, which might be important in some tasks.

What does lowercasing do to the word 'Hello'?

A Removes the word

B Converts it to 'HELLO'

C Converts it to 'hello'

D Adds punctuation

Lowercasing changes all letters to lowercase, so 'Hello' becomes 'hello'.

Which of these is NOT a normalization step?

A Adding random characters

B Lowercasing

C Removing accents

D Replacing multiple spaces with one

Adding random characters is not normalization; normalization cleans and standardizes text.

Why normalize text before training an NLP model?

A To increase text length

B To make text consistent and easier to understand

C To add noise to data

D To remove all vowels

Normalization makes text consistent, helping the model learn better.

What is a common effect of lowercasing on vocabulary size?

A Vocabulary size increases

B Vocabulary size doubles

C Vocabulary size stays the same

D Vocabulary size decreases

Lowercasing merges words differing only by case, reducing vocabulary size.

Which is a risk of lowercasing text?

A Losing important case information

B Making text longer

C Adding accents

D Removing stopwords

Lowercasing can lose case information like proper nouns or acronyms.

Explain why lowercasing and normalization are important in preparing text for machine learning models.

Think about how text variations affect model learning.

Describe some common normalization techniques used in NLP besides lowercasing.

Consider how text can be made consistent.