Multilingual models in NLP - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is a multilingual model in AI?

A multilingual model is an AI system trained to understand and generate text in multiple languages, allowing it to work across different languages without needing separate models for each.

beginner

Why are multilingual models useful in real life?

They help people communicate and access information across languages, like translating websites or understanding customer feedback from different countries, all with one model.

intermediate

How do multilingual models learn multiple languages?

They are trained on large collections of text from many languages, learning patterns and meanings that apply across languages, so they can understand and generate text in each one.

intermediate

What is a challenge when building multilingual models?

One challenge is balancing performance: the model might do well in popular languages but less well in rare ones because it sees less data for those languages during training.

beginner

Name a popular multilingual model used in natural language processing.

One popular multilingual model is mBERT (multilingual BERT), which supports over 100 languages and is widely used for tasks like translation and text classification.

What does a multilingual model do?

A Processes multiple languages with one model

B Uses one model per language

C Only translates text

D Works only with English text

Multilingual models handle many languages using a single model, unlike separate models for each language.

Which is a common training method for multilingual models?

A Training on one language at a time separately

B Training only on English text

C Training on images only

D Training on text from many languages together

Multilingual models learn by training on mixed text data from many languages simultaneously.

What is a limitation of multilingual models?

A They only work with numbers

B They may perform worse on less common languages

C They cannot translate text

D They require separate models per language

Multilingual models often have less data for rare languages, which can reduce their accuracy for those languages.

Which model is an example of a multilingual model?

A VGG16

B ResNet

C mBERT

D GPT-1 English only

mBERT is a well-known multilingual model supporting many languages.

Why might companies use multilingual models?

A To handle customer support in many languages with one system

B To only work with English customers

C To avoid using AI

D To translate images

Multilingual models help companies serve customers speaking different languages efficiently.

Explain what a multilingual model is and why it is important.

Think about how one model can work with many languages.

Describe one challenge faced when training multilingual models and how it affects performance.

Consider how the amount of data per language matters.

Practice

1. What is the main advantage of using a multilingual model in natural language processing?

easy

A. It can understand and process multiple languages with a single model.

B. It requires training a separate model for each language.

C. It only works for English language tasks.

D. It uses more resources than training individual models.

Solution

1. Step 1: Understand the purpose of multilingual models

   Multilingual models are designed to handle many languages using one model instead of separate ones.

2. Step 2: Compare advantages

   This approach saves time and resources by avoiding multiple models for different languages.

3. Final Answer:

   It can understand and process multiple languages with a single model. -> Option A

4. Quick Check:

   Multilingual model advantage = single model for many languages [OK]

Hint: Multilingual means one model for many languages [OK]

Common Mistakes:

• Thinking multilingual models only work for English
• Assuming separate models are needed per language
• Believing multilingual models use more resources

2. Which of the following is the correct way to load a multilingual model using Hugging Face Transformers in Python?

easy

A. model = AutoModel.from_pretrained('xlm-roberta-base')

B. model = AutoModel.from_pretrained('gpt2')

C. model = AutoModel.from_pretrained('bert-base-uncased')

D. model = AutoModel.from_pretrained('bert-large-cased')

Solution

1. Step 1: Identify multilingual model names

   'xlm-roberta-base' is a well-known multilingual model supporting many languages.

2. Step 2: Check other options

   'bert-base-uncased' and 'bert-large-cased' are English-only models; 'gpt2' is a generative English model.

3. Final Answer:

   model = AutoModel.from_pretrained('xlm-roberta-base') -> Option A

4. Quick Check:

   Multilingual model name = 'xlm-roberta-base' [OK]

Hint: Look for 'xlm' or 'multilingual' in model name [OK]

Common Mistakes:

• Choosing English-only models for multilingual tasks
• Confusing generative models with multilingual encoders
• Using model names without checking language support

3. Consider this Python code using Hugging Face Transformers:

from transformers import AutoTokenizer, AutoModelForSequenceClassification

tokenizer = AutoTokenizer.from_pretrained('xlm-roberta-base')
model = AutoModelForSequenceClassification.from_pretrained('xlm-roberta-base')
inputs = tokenizer('Bonjour, comment ça va?', return_tensors='pt')
outputs = model(**inputs)
print(outputs.logits.shape)

What will be the printed output shape?

medium

A. torch.Size([1, 1])

B. torch.Size([1, 2])

C. torch.Size([1, 512])

D. torch.Size([1, 768])

Solution

1. Step 1: Understand model type and output

   The model is for sequence classification, which outputs logits for each class. The default 'xlm-roberta-base' classification head has 2 classes.

2. Step 2: Determine output shape

   Batch size is 1 (one sentence), so output logits shape is [1, 2].

3. Final Answer:

   torch.Size([1, 2]) -> Option B

4. Quick Check:

   Sequence classification logits shape = [batch, classes] = [1, 2] [OK]

Hint: Classification logits shape = batch size x number of classes [OK]

Common Mistakes:

• Confusing hidden size with output logits shape
• Assuming output shape matches input token length
• Ignoring batch size dimension

4. You tried to use a multilingual model but got this error:
ValueError: Tokenizer does not have a pad token.
What is the best way to fix this error?

medium

A. Use a different model that does not require padding.

B. Add padding=True when calling the tokenizer.

C. Manually set the pad token with tokenizer.pad_token = tokenizer.eos_token.

D. Ignore the error and continue training.

Solution

1. Step 1: Understand the error cause

   The tokenizer lacks a pad token, which is needed to pad sequences to the same length.

2. Step 2: Fix by assigning pad token

   Assigning the pad token to an existing token like eos_token solves the issue.

3. Final Answer:

   Manually set the pad token with tokenizer.pad_token = tokenizer.eos_token. -> Option C

4. Quick Check:

   Set pad token manually to fix padding error [OK]

Hint: Set pad token manually if missing in tokenizer [OK]

Common Mistakes:

• Ignoring padding requirement
• Trying to skip padding without fixing tokenizer
• Switching models unnecessarily

5. You want to build a multilingual sentiment analysis system supporting English, Spanish, and French. Which approach best balances accuracy and resource use?

hard

A. Train separate models for each language from scratch.

B. Use a rule-based system with language-specific sentiment dictionaries.

C. Use an English-only model and translate all inputs to English before analysis.

D. Use a single pretrained multilingual model fine-tuned on combined data from all three languages.

Solution

1. Step 1: Consider resource and accuracy trade-offs

   Training separate models is resource-heavy; rule-based systems lack accuracy; translation adds errors.

2. Step 2: Choose multilingual fine-tuning

   Fine-tuning one multilingual pretrained model on combined data leverages shared knowledge and saves resources.

3. Final Answer:

   Use a single pretrained multilingual model fine-tuned on combined data from all three languages. -> Option D

4. Quick Check:

   Multilingual fine-tuning balances accuracy and efficiency [OK]

Hint: Fine-tune one multilingual model on all languages together [OK]

Common Mistakes:

• Training separate models wastes resources
• Relying on translation reduces accuracy
• Using rule-based methods limits performance