GPT family overview in NLP - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to import the GPT model from the transformers library.

NLP

from transformers import [1]

Drag options to blanks, or click blank then click option'

AGPT2Model

BBertModel

CRobertaModel

DT5Model

Attempts:

3 left

💡 Hint

Common Mistakes

Importing BertModel or RobertaModel which are different model families.

Using T5Model which is unrelated to GPT.

The GPT family models are accessed via GPT2Model in the transformers library.

2fill in blank

medium

Complete the code to load the GPT-2 tokenizer.

NLP

from transformers import GPT2Tokenizer

tokenizer = GPT2Tokenizer.from_pretrained([1])

Drag options to blanks, or click blank then click option'

A't5-small'

B'gpt2'

C'roberta-base'

D'bert-base-uncased'

Attempts:

3 left

💡 Hint

Common Mistakes

Using tokenizer names from other model families like BERT or RoBERTa.

Forgetting the quotes around the model name.

The GPT-2 tokenizer is loaded using the model name 'gpt2'.

3fill in blank

hard

Fix the error in the code to generate text using GPT-2.

NLP

from transformers import GPT2LMHeadModel, GPT2Tokenizer

tokenizer = GPT2Tokenizer.from_pretrained('gpt2')
model = GPT2LMHeadModel.from_pretrained('gpt2')

input_ids = tokenizer.encode('Hello, how are you?', return_tensors=[1])
outputs = model.generate(input_ids, max_length=20)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Drag options to blanks, or click blank then click option'

Apt

Btf

Cnp

Dtorch

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'tf' which is for TensorFlow tensors.

Using 'torch' which is not a valid string argument here.

The return_tensors argument expects 'pt' for PyTorch tensors when using GPT-2 with transformers.

4fill in blank

hard

Fill both blanks to create a dictionary comprehension that maps GPT model names to their sizes in millions of parameters.

NLP

gpt_sizes = { [1] : [2] for [1] in ['gpt2', 'gpt2-medium', 'gpt2-large'] }

Drag options to blanks, or click blank then click option'

Amodel

B{'gpt2': 124, 'gpt2-medium': 355, 'gpt2-large': 774}

Csize

Dparams

Attempts:

3 left

💡 Hint

Common Mistakes

Using the same variable name for key and value incorrectly.

Trying to use a variable that is not defined in the comprehension.

The variable model is used as the key, and the dictionary with sizes is used as the value for each key in the comprehension.

5fill in blank

hard

Fill both blanks to create a function that returns the number of tokens in a text using GPT-2 tokenizer.

NLP

def count_tokens(text):
    tokenizer = GPT2Tokenizer.from_pretrained([1])
    tokens = tokenizer.[2](text)
    return len(tokens)  # number of tokens

print(count_tokens('Hello world!'))

Drag options to blanks, or click blank then click option'

A'gpt2'

Bencode

Ctokenize

D'bert-base-uncased'

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'tokenize' instead of 'encode'.

Loading tokenizer with a wrong model name.

The GPT-2 tokenizer is loaded with 'gpt2'. The method to convert text to tokens is encode. The function returns the length of the token list.