Vision-language models (GPT-4V) in Prompt Engineering / GenAI - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to load a GPT-4V vision-language model.

Prompt Engineering / GenAI

model = GPT4VModel.from_pretrained([1])

Drag options to blanks, or click blank then click option'

A"gpt4v-text"

B"gpt4v-base"

C"gpt4v-vision"

D"gpt4v-audio"

Attempts:

3 left

💡 Hint

Common Mistakes

Using a text-only model name instead of the vision-language model.

Confusing audio or base models with vision-language models.

The correct model identifier for the GPT-4V vision-language model is "gpt4v-vision".

2fill in blank

medium

Complete the code to preprocess an image for GPT-4V input.

Prompt Engineering / GenAI

processed_image = processor.preprocess([1])

Drag options to blanks, or click blank then click option'

Araw_text

Bimage_path

Caudio_clip

Dvideo_frame

Attempts:

3 left

💡 Hint

Common Mistakes

Passing raw text or audio instead of an image.

Using a video frame variable without conversion.

Images must be loaded from a path or image object before preprocessing for GPT-4V.

3fill in blank

hard

Fix the error in the code to generate a caption from an image using GPT-4V.

Prompt Engineering / GenAI

outputs = model.generate([1])

Drag options to blanks, or click blank then click option'

Aaudio_input

Braw_text

Cvideo_input

Dprocessed_image

Attempts:

3 left

💡 Hint

Common Mistakes

Passing raw text or audio instead of processed image tensor.

Using unprocessed image data.

The model's generate method requires the processed image tensor as input, not raw text or other media.

4fill in blank

hard

Fill both blanks to create a dictionary of images and their features.

Prompt Engineering / GenAI

features = {img: [1] for img in images if img [2] None}

Drag options to blanks, or click blank then click option'

Amodel.extract_features(img)

Bmodel.generate_caption(img)

Cis not

D==

Attempts:

3 left

💡 Hint

Common Mistakes

Using '==' instead of 'is not' to check for None.

Calling generate_caption instead of extract_features.

We extract features from each image that is not None using model.extract_features and check with 'is not'.

5fill in blank

hard

Fill all three blanks to filter images and generate captions for valid inputs.

Prompt Engineering / GenAI

captions = [model.generate_caption([1]) for [2] in images if [3] is not None]

Drag options to blanks, or click blank then click option'

Aimg

Bimage

Attempts:

3 left

💡 Hint

Common Mistakes

Mixing variable names inconsistently.

Checking the wrong variable for None.

We iterate over images using 'image' as the variable, generate captions for each 'image', and check 'image is not None'.