Embedding layers in TensorFlow - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to create an embedding layer with input dimension 1000 and output dimension 64.

TensorFlow

embedding_layer = tf.keras.layers.Embedding(input_dim=1000, output_dim=[1])

Drag options to blanks, or click blank then click option'

A32

B128

C64

D256

Attempts:

3 left

💡 Hint

Common Mistakes

Confusing input_dim and output_dim values.

Using output_dim larger than necessary.

The output_dim parameter defines the size of the embedding vectors. Here, 64 is the correct size.

2fill in blank

medium

Complete the code to define an embedding layer that accepts input sequences of length 10.

TensorFlow

embedding_layer = tf.keras.layers.Embedding(input_dim=5000, output_dim=50, input_length=[1])

Drag options to blanks, or click blank then click option'

A10

B20

C5

D50

Attempts:

3 left

💡 Hint

Common Mistakes

Setting input_length to output_dim by mistake.

Using input_length larger than actual sequence length.

The input_length parameter specifies the length of input sequences. Here, it is 10.

3fill in blank

hard

Fix the error in the embedding layer creation by filling the correct input_dim.

TensorFlow

embedding_layer = tf.keras.layers.Embedding(input_dim=[1], output_dim=100)

Drag options to blanks, or click blank then click option'

A0

B10000

C-1

D100

Attempts:

3 left

💡 Hint

Common Mistakes

Using zero or negative numbers for input_dim.

Confusing input_dim with output_dim.

input_dim must be a positive integer representing vocabulary size. 10000 is a valid size; 0 or negative values cause errors.

4fill in blank

hard

Fill both blanks to create an embedding layer and get the output shape after passing input of length 7.

TensorFlow

embedding_layer = tf.keras.layers.Embedding(input_dim=[1], output_dim=[2], input_length=7)
output_shape = embedding_layer.compute_output_shape((None, 7))

Drag options to blanks, or click blank then click option'

A5000

B64

C128

D10000

Attempts:

3 left

💡 Hint

Common Mistakes

Mixing up input_dim and output_dim values.

Using input_length as a blank instead of fixed 7.

input_dim is 5000 for vocabulary size, output_dim is 64 for embedding size. The output shape will be (None, 7, 64).

5fill in blank

hard

Fill all three blanks to create an embedding layer, pass input data, and get the shape of the output tensor.

TensorFlow

embedding_layer = tf.keras.layers.Embedding(input_dim=[1], output_dim=[2], input_length=5)
input_data = tf.constant([[1, 2, 3, 4, 5]])
output = embedding_layer([3])
output_shape = output.shape

Drag options to blanks, or click blank then click option'

A1000

B32

Cinput_data

Dtf.constant([[0, 1, 2, 3, 4]])

Attempts:

3 left

💡 Hint

Common Mistakes

Passing wrong tensor to embedding layer.

Confusing input_dim and output_dim values.

input_dim is 1000, output_dim is 32, and the input tensor is input_data. The output shape will be (1, 5, 32).

Practice

1. What is the main purpose of an Embedding layer in TensorFlow?

easy

A. To reduce the dimensionality of numerical data using PCA

B. To normalize input data between 0 and 1

C. To perform convolution operations on images

D. To convert categorical data or words into dense vectors of numbers

Solution

1. Step 1: Understand the role of embedding layers

   Embedding layers transform categorical or text data into vectors that capture relationships.

2. Step 2: Compare with other options

   Normalization, convolution, and PCA are different processes unrelated to embedding layers.

3. Final Answer:

   To convert categorical data or words into dense vectors of numbers -> Option D

4. Quick Check:

   Embedding layer = vector conversion [OK]

Hint: Embedding layers turn categories into number vectors [OK]

Common Mistakes:

• Confusing embedding with normalization
• Thinking embedding is for images
• Mixing embedding with dimensionality reduction

2. Which of the following is the correct way to create an embedding layer in TensorFlow for 1000 words with 64-dimensional vectors?

easy

A. tf.keras.layers.Embedding(input_dim=1000, output_dim=64)

B. tf.keras.layers.Embedding(input_dim=64, output_dim=1000)

C. tf.keras.layers.Dense(1000, 64)

D. tf.keras.layers.Embedding(1000, 64, input_length=1000)

Solution

1. Step 1: Identify correct parameters for Embedding

   Embedding requires input_dim as vocabulary size and output_dim as vector size.

2. Step 2: Check each option

   tf.keras.layers.Embedding(input_dim=1000, output_dim=64) correctly uses input_dim=1000 and output_dim=64. tf.keras.layers.Embedding(input_dim=64, output_dim=1000) swaps them. tf.keras.layers.Dense(1000, 64) uses Dense layer incorrectly. tf.keras.layers.Embedding(1000, 64, input_length=1000) wrongly sets input_length to 1000, which is usually sequence length, not vocabulary size.

3. Final Answer:

   tf.keras.layers.Embedding(input_dim=1000, output_dim=64) -> Option A

4. Quick Check:

   input_dim = vocab size, output_dim = vector size [OK]

Hint: input_dim = vocab size, output_dim = embedding size [OK]

Common Mistakes:

• Swapping input_dim and output_dim
• Using Dense layer instead of Embedding
• Confusing input_length with input_dim

3. What will be the output shape of the embedding layer for input shape (batch_size=32, sequence_length=10) if the embedding layer is defined as Embedding(input_dim=5000, output_dim=128)?

medium

A. (32, 10, 5000)

B. (32, 128, 10)

C. (32, 10, 128)

D. (10, 32, 128)

Solution

1. Step 1: Understand input and output shapes of Embedding

   Embedding converts each integer in the input sequence to a vector of length output_dim. Input shape is (batch_size, sequence_length).

2. Step 2: Calculate output shape

   For input shape (32, 10), output shape becomes (32, 10, 128) because each of the 10 tokens is replaced by a 128-dimensional vector.

3. Final Answer:

   (32, 10, 128) -> Option C

4. Quick Check:

   Output shape = (batch, sequence, embedding size) [OK]

Hint: Output shape adds embedding size as last dimension [OK]

Common Mistakes:

• Confusing input_dim with output shape
• Swapping batch and sequence dimensions
• Ignoring embedding vector size in output

4. You wrote this code but get an error:

embedding = tf.keras.layers.Embedding(input_dim=1000, output_dim=64)
input_data = tf.constant([[1, 2, 3], [4, 5, 1001]])
output = embedding(input_data)

What is the cause of the error?

medium

A. The input contains an index (1001) outside the vocabulary size

B. The output_dim is too large for the input

C. Embedding layer requires input shape to be 1D

D. The input data type must be float, not int

Solution

1. Step 1: Check input indices against input_dim

   Embedding input indices must be in range [0, input_dim-1]. Here, input_dim=1000, so max index is 999.

2. Step 2: Identify invalid index

   Input contains 1001, which is out of range and causes an error.

3. Final Answer:

   The input contains an index (1001) outside the vocabulary size -> Option A

4. Quick Check:

   Input indices must be less than input_dim [OK]

Hint: Check input indices are less than input_dim [OK]

Common Mistakes:

• Ignoring out-of-range indices
• Thinking input must be 1D
• Confusing data types for embedding input

5. You want to use an embedding layer for a text classification model. Your vocabulary size is 5000, and you want embeddings of size 50. You also want to limit the input sequences to length 20. Which of the following code snippets correctly creates the embedding layer and input shape for this model?

hard

A. embedding = tf.keras.layers.Embedding(input_dim=5000, output_dim=20, input_length=50)

B. embedding = tf.keras.layers.Embedding(input_dim=5000, output_dim=50, input_length=20)

C. embedding = tf.keras.layers.Embedding(input_dim=20, output_dim=50, input_length=5000)

D. embedding = tf.keras.layers.Embedding(input_dim=50, output_dim=5000, input_length=20)

Solution

1. Step 1: Match parameters to problem requirements

   Vocabulary size is 5000, so input_dim=5000. Embedding size is 50, so output_dim=50. Input sequences length is 20, so input_length=20.

2. Step 2: Validate each option

   embedding = tf.keras.layers.Embedding(input_dim=5000, output_dim=50, input_length=20) matches all requirements correctly. Options A, B, and D have incorrect parameter assignments.

3. Final Answer:

   embedding = tf.keras.layers.Embedding(input_dim=5000, output_dim=50, input_length=20) -> Option B

4. Quick Check:

   input_dim = vocab, output_dim = embed size, input_length = seq length [OK]

Hint: Match input_dim, output_dim, input_length to vocab, embed size, seq length [OK]

Common Mistakes:

• Swapping input_dim and input_length
• Confusing output_dim with input_length
• Using wrong values for embedding size