Model selection for tasks in NLP - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Model Choice

intermediate

Choosing the best model for sentiment analysis

You want to build a model that classifies movie reviews as positive or negative. Which model is the best choice for this binary text classification task?

A A convolutional neural network (CNN) designed for image recognition

B A recurrent neural network (RNN) or transformer-based model trained on text data

C A k-means clustering algorithm

D A linear regression model

💡 Hint

Think about models that understand sequences of words and context in text.

RNNs and transformer models are designed to process sequences like text and capture context, making them suitable for sentiment analysis. CNNs for images, clustering, and regression are not appropriate for this task.

❓ Metrics

intermediate

Evaluating a multi-class text classifier

You trained a model to classify news articles into 5 categories. Which metric is best to evaluate overall model performance?

A Accuracy

B Mean Squared Error (MSE)

C BLEU score

D Perplexity

💡 Hint

Consider a metric that measures correct predictions over total predictions for classification.

Accuracy measures the proportion of correct predictions in classification tasks. MSE is for regression, BLEU is for language generation quality, and perplexity measures language model uncertainty.

🧠 Conceptual

advanced

Selecting a model for named entity recognition (NER)

You need to identify names of people, places, and organizations in text. Which model type is most suitable for this sequence labeling task?

A Transformer-based model with token classification head

B Feedforward neural network with bag-of-words input

C K-nearest neighbors classifier

D Autoencoder for dimensionality reduction

💡 Hint

NER requires understanding each word's role in context within a sentence.

Transformer models with token classification heads can label each word in a sequence, making them ideal for NER. Feedforward networks lose sequence order, KNN is not suited for sequence labeling, and autoencoders are for unsupervised tasks.

🔧 Debug

advanced

Why does this text classification model perform poorly?

You trained a text classifier using a bag-of-words model and logistic regression. The accuracy is very low on test data. What is the most likely reason?

A The model is overfitting because it has too many parameters

B The dataset is too large for logistic regression to train

C Bag-of-words ignores word order and context, limiting performance on complex language tasks

D Logistic regression cannot handle binary classification

💡 Hint

Think about what information bag-of-words loses about the text.

Bag-of-words treats text as unordered word counts, losing context and word order, which are important for understanding meaning. Logistic regression can handle binary classification and large datasets, and overfitting usually increases training accuracy.

❓ Hyperparameter

expert

Choosing hyperparameters for fine-tuning a transformer model

You are fine-tuning a pre-trained transformer model on a small labeled dataset for text classification. Which hyperparameter setting is most important to avoid overfitting?

A Train for many epochs without early stopping

B Use a very high learning rate (e.g., 0.1) to speed up training

C Freeze all layers and only train the final classification layer

D Use a small batch size and apply dropout regularization

💡 Hint

Think about techniques that reduce overfitting when data is limited.

Small batch sizes and dropout help regularize training and reduce overfitting. High learning rates can cause unstable training. Training many epochs without stopping risks overfitting. Freezing layers can help but may limit learning capacity.

Practice

1. Which type of model is best suited for a task where you want to classify emails as spam or not spam?

easy

A. A regression model

B. A clustering model

C. A classification model

D. A dimensionality reduction model

Solution

1. Step 1: Understand the task type

   Classifying emails as spam or not spam is a task where the output is a category or label.

2. Step 2: Match task to model type

   Classification models are designed to assign categories to inputs, so they fit this task best.

3. Final Answer:

   A classification model -> Option C

4. Quick Check:

   Classification = Spam detection [OK]

Hint: Classification = sorting into groups or labels [OK]

Common Mistakes:

• Confusing classification with regression
• Choosing clustering which is unsupervised
• Selecting dimensionality reduction which is for feature simplification

2. Which of the following is the correct way to import a pre-trained BERT model using the Hugging Face Transformers library in Python?

easy

A. from transformers import BertModel; model = BertModel.from_pretrained('bert-base-uncased')

B. import bert; model = bert.load('bert-base-uncased')

C. from transformers import BERT; model = BERT.load_pretrained('bert-base-uncased')

D. import transformers; model = transformers.Bert.load('bert-base-uncased')

Solution

1. Step 1: Recall correct import syntax

   The Hugging Face library uses 'from transformers import BertModel' to import the BERT model class.

2. Step 2: Use the correct method to load pre-trained weights

   The method to load pre-trained weights is 'from_pretrained' called on the model class.

3. Final Answer:

   from transformers import BertModel; model = BertModel.from_pretrained('bert-base-uncased') -> Option A

4. Quick Check:

   Correct import and method = from transformers import BertModel; model = BertModel.from_pretrained('bert-base-uncased') [OK]

Hint: Use 'from transformers import ModelName' and 'from_pretrained' method [OK]

Common Mistakes:

• Using incorrect class names like BERT instead of BertModel
• Calling non-existent methods like load_pretrained
• Wrong import statements or missing 'from'

3. Consider the following Python code using scikit-learn for sentiment analysis. What will be the printed accuracy after training and testing the model?

from sklearn.feature_extraction.text import CountVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split

texts = ['I love this movie', 'This movie is bad', 'Amazing film', 'Terrible acting', 'Great plot', 'Worst movie ever']
labels = [1, 0, 1, 0, 1, 0]

X_train, X_test, y_train, y_test = train_test_split(texts, labels, test_size=0.33, random_state=42)

vectorizer = CountVectorizer()
X_train_vec = vectorizer.fit_transform(X_train)
X_test_vec = vectorizer.transform(X_test)

model = LogisticRegression()
model.fit(X_train_vec, y_train)

accuracy = model.score(X_test_vec, y_test)
print(round(accuracy, 2))

medium

A. 0.67

B. 1.0

C. 0.33

D. 0.5

Solution

1. Step 1: Understand data split and labels

   With random_state=42 and test_size=0.33, the test set contains 2 samples: 'Amazing film' (label 1) and 'Worst movie ever' (label 0).

2. Step 2: Predict and check accuracy

   Logistic Regression on simple text features predicts one sample correctly and one incorrectly, resulting in accuracy 0.5.

3. Final Answer:

   0.5 -> Option D

4. Quick Check:

   Accuracy = 1 correct / 2 total = 0.5 [OK]

Hint: Accuracy = correct predictions / total test samples [OK]

Common Mistakes:

• Assuming perfect accuracy without checking data split
• Confusing training accuracy with test accuracy
• Ignoring random_state effect on split

4. You tried to train a text classification model but got an error: ValueError: Found input variables with inconsistent numbers of samples. What is the most likely cause?

medium

A. The learning rate is too high

B. The input texts and labels have different lengths

C. The model architecture is incorrect

D. The tokenizer is missing

Solution

1. Step 1: Understand the error message

   The error says input variables have inconsistent sample counts, meaning the number of texts and labels differ.

2. Step 2: Identify cause in data preparation

   If texts and labels lists have different lengths, model training fails because each input needs a matching label.

3. Final Answer:

   The input texts and labels have different lengths -> Option B

4. Quick Check:

   Mismatch in input and label counts causes this error [OK]

Hint: Check if inputs and labels have same number of samples [OK]

Common Mistakes:

• Blaming model architecture for data size errors
• Ignoring data preprocessing steps
• Assuming learning rate or tokenizer causes this error

5. You want to build a model to generate text summaries from long articles. Which model type should you select for this task?

hard

A. Sequence-to-sequence model with attention

B. Binary classification model

C. Clustering model

D. Regression model

Solution

1. Step 1: Understand the task requirements

   Text summarization requires converting a long sequence (article) into a shorter sequence (summary).

2. Step 2: Identify suitable model architecture

   Sequence-to-sequence models with attention are designed to handle input and output sequences of different lengths effectively.

3. Final Answer:

   Sequence-to-sequence model with attention -> Option A

4. Quick Check:

   Summarization = sequence-to-sequence with attention [OK]

Hint: Summarization needs sequence-to-sequence models [OK]

Common Mistakes:

• Choosing classification or regression which don't generate sequences
• Selecting clustering which is unsupervised grouping
• Ignoring the need for attention in long text tasks