First NLP pipeline - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is the first step in a typical NLP pipeline?

The first step is usually text preprocessing, which includes cleaning the text by removing unwanted characters, converting text to lowercase, and tokenizing sentences into words.

beginner

What does tokenization mean in NLP?

Tokenization means splitting text into smaller pieces called tokens, usually words or sentences, to make it easier for the computer to understand and analyze the text.

beginner

Why do we remove stop words in an NLP pipeline?

Stop words are common words like 'the', 'is', and 'and' that usually do not add much meaning. Removing them helps the model focus on important words and improves efficiency.

intermediate

What is lemmatization in an NLP pipeline?

Lemmatization is the process of converting words to their base or dictionary form, like changing 'running' to 'run', to treat different forms of a word as the same.

intermediate

Name the main components of a simple NLP pipeline.

A simple NLP pipeline usually includes:

• Text preprocessing (cleaning, tokenization)
• Stop word removal
• Lemmatization or stemming
• Feature extraction (like bag of words or embeddings)
• Model training or prediction

What is the purpose of tokenization in an NLP pipeline?

A Convert text to uppercase

B Remove punctuation from text

C Split text into smaller units like words or sentences

D Train the machine learning model

Tokenization breaks text into smaller pieces called tokens, which helps the computer analyze the text.

Which step removes common words like 'and', 'the', and 'is'?

A Stop word removal

B Lemmatization

C Tokenization

D Feature extraction

Stop word removal filters out common words that usually do not add meaning.

What does lemmatization do in an NLP pipeline?

A Splits text into sentences

B Converts words to their base form

C Removes punctuation

D Counts word frequency

Lemmatization converts words like 'running' to 'run' to unify word forms.

Which of these is NOT usually part of the first NLP pipeline steps?

A Text cleaning

B Tokenization

C Stop word removal

D Model training

Model training usually comes after preprocessing steps like cleaning and tokenization.

Why do we preprocess text in NLP?

A To prepare text for analysis by cleaning and structuring it

B To make text harder to understand

C To add random noise to data

D To translate text into another language

Preprocessing cleans and structures text so models can analyze it better.

Describe the main steps involved in a first NLP pipeline and why each step is important.

Think about how raw text is prepared for a computer to understand.

Explain how tokenization and lemmatization help improve text analysis in NLP.

Consider how breaking down and simplifying words helps machines.