Embedding layer usage in NLP - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Output of embedding layer with given input indices

What is the shape of the output tensor after passing the input indices through the embedding layer?

NLP

import torch
import torch.nn as nn

embedding = nn.Embedding(num_embeddings=10, embedding_dim=4)
input_indices = torch.tensor([1, 3, 7, 9])
output = embedding(input_indices)
output_shape = output.shape
print(output_shape)

A torch.Size([1, 4])

B torch.Size([4])

C torch.Size([4, 4])

D torch.Size([10, 4])

💡 Hint

The embedding layer converts each index into a vector of the embedding dimension.

The input tensor has 4 indices, and each index is converted into a vector of size 4, so the output shape is (4, 4).

❓ Model Choice

intermediate

Choosing embedding layer parameters for vocabulary size and embedding dimension

You want to create an embedding layer for a vocabulary of 5000 words, each represented by a 50-dimensional vector. Which is the correct way to initialize this embedding layer in PyTorch?

A nn.Embedding(num_embeddings=50, embedding_dim=5000)

B nn.Embedding(num_embeddings=5000, embedding_dim=50)

C nn.Embedding(num_embeddings=5000, embedding_dim=5000)

D nn.Embedding(num_embeddings=50, embedding_dim=50)

💡 Hint

The first parameter is the vocabulary size, the second is the vector size.

The embedding layer's first argument is the number of unique tokens (vocabulary size), and the second is the size of each embedding vector.

❓ Hyperparameter

advanced

Effect of embedding dimension size on model performance

Increasing the embedding dimension size in a neural network model typically results in which of the following?

A Higher model capacity but increased risk of overfitting

B Lower model capacity and faster training

C No change in model capacity or training time

D Guaranteed better generalization on unseen data

💡 Hint

Think about how larger embeddings affect the number of parameters.

Larger embedding dimensions increase the number of parameters, which can improve capacity but also risk overfitting if not managed.

🔧 Debug

advanced

Identifying error in embedding layer input

What error will this code raise when running the embedding layer with the given input? import torch import torch.nn as nn embedding = nn.Embedding(10, 3) input_indices = torch.tensor([1.0, 2.0, 3.0]) output = embedding(input_indices)

A No error, runs successfully

B TypeError: embedding layer expects a list, not a tensor

C ValueError: embedding dimension mismatch

D RuntimeError: 'input' must be a tensor of dtype torch.int64

💡 Hint

Embedding layers require integer indices, not floats.

Embedding layers require input tensors of integer type (torch.int64). Passing floats causes a runtime error.

🧠 Conceptual

expert

Why use pretrained embeddings instead of training from scratch?

Which of the following is the main advantage of using pretrained word embeddings in a natural language processing model?

A They provide rich semantic information learned from large corpora, improving model performance especially with limited data

B They reduce the model size by compressing the vocabulary

C They eliminate the need for any further training or fine-tuning

D They guarantee 100% accuracy on all NLP tasks

💡 Hint

Think about what pretrained embeddings capture from language data.

Pretrained embeddings capture semantic relationships from large datasets, helping models generalize better when training data is limited.