Language modeling concept in NLP - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

What does a language model predict?

In simple terms, what is the main task of a language model?

A Predict the next word in a sentence based on previous words

B Translate text from one language to another

C Identify the topic of a given text

D Summarize a long paragraph into a short sentence

💡 Hint

Think about what comes next when you read a sentence.

A language model learns to guess the next word by looking at the words before it. This helps computers understand and generate text.

❓ Predict Output

intermediate

Output of a simple bigram language model prediction

Given the following bigram probabilities, what is the predicted next word after 'I'?

{'I': {'am': 0.6, 'like': 0.3, 'have': 0.1}}

A 'is'

B 'like'

C 'have'

D 'am'

💡 Hint

Choose the word with the highest probability after 'I'.

The word 'am' has the highest probability (0.6) following 'I', so it is the predicted next word.

❓ Model Choice

advanced

Choosing a model type for language modeling

You want to build a language model that understands long sentences and context. Which model type is best?

A N-gram model with n=2

B Recurrent Neural Network (RNN)

C Bag-of-words model

D Simple linear regression

💡 Hint

Think about models that remember previous words in a sequence.

RNNs can remember information from earlier words, making them good for understanding long sentences and context.

❓ Hyperparameter

advanced

Effect of increasing vocabulary size in language models

What is a likely effect of increasing the vocabulary size in a language model?

A Model requires more memory and training data

B Model training becomes faster and simpler

C Model accuracy always decreases

D Model ignores rare words automatically

💡 Hint

Think about what happens when the model has to learn more words.

More vocabulary means more parameters and data needed to learn word patterns well, increasing memory and data requirements.

❓ Metrics

expert

Interpreting perplexity in language models

A language model has a perplexity score of 20 on a test set. What does this mean?

A The model is 20 times better than random guessing

B The model makes 20% errors in predictions

C On average, the model is as uncertain as choosing among 20 options

D The model predicts the next word with 20% probability

💡 Hint

Perplexity measures how surprised the model is by the test data.

Perplexity of 20 means the model is as uncertain as picking one word out of 20 equally likely choices on average.