GenAI applications (text, image, code, audio) - Model Pipeline Trace

Model Pipeline - GenAI applications (text, image, code, audio)

This pipeline shows how Generative AI models create new content like text, images, code, or audio from input prompts. It starts with input data, processes it, trains a model, and then generates creative outputs.

Data Flow - 5 Stages

1Input Prompt

1 prompt string→User provides a text prompt describing desired output→1 prompt string

"Draw a sunset over mountains"

↓

2Preprocessing

1 prompt string→Convert prompt to tokens or embeddings for model input→1 sequence of tokens (e.g., 20 tokens)

["Draw", "a", "sunset", "over", "mountains"]

↓

3Model Training

Millions of prompt-output pairs→Train a generative model (e.g., transformer) to learn patterns→Trained model weights

Model learns how to generate images from text prompts

↓

4Generation

1 prompt token sequence→Model generates output tokens step-by-step→Generated content tokens (text, image pixels, code lines, audio frames)

Generated image pixels forming a sunset scene

↓

5Postprocessing

Generated tokens→Convert tokens to human-readable or viewable format→Final output (text string, image file, code snippet, audio clip)

PNG image of sunset over mountains

Training Trace - Epoch by Epoch

Loss
2.5 |****
2.0 |***
1.5 |**
1.0 |**
0.5 |*
0.0 +----------------
      1  5 10 15 20 Epochs

Epoch	Loss ↓	Accuracy ↑	Observation
1	2.3	0.15	Model starts learning basic patterns from data
5	1.2	0.45	Model improves understanding of prompt-output relations
10	0.7	0.70	Model generates more coherent and relevant outputs
15	0.4	0.85	Model produces high-quality content with fewer errors
20	0.25	0.92	Model converges with strong generation ability

Prediction Trace - 5 Layers

Layer 1: Tokenization

Layer 2: Embedding Layer

Layer 3: Transformer Layers

Layer 4: Decoder Generation

Layer 5: Detokenization

Model Quiz - 3 Questions

What is the first step in the GenAI pipeline after receiving a user prompt?

A Model training

B Postprocessing output

C Tokenization of the prompt

D Generating final content

Key Insight

Generative AI models learn from many examples to create new content by understanding input prompts and generating outputs step-by-step. Training reduces errors and improves output quality over time.

Practice

1. Which of the following is NOT a common application of GenAI?

easy

A. Manually coding software without AI help

B. Creating images from simple descriptions

C. Automatically generating text like stories or emails

D. Producing audio like music or speech

Solution

1. Step 1: Understand GenAI applications

   GenAI is used to create text, images, code, and audio automatically from prompts.

2. Step 2: Identify the option that does not involve AI

   Manual coding without AI help is not an application of GenAI.

3. Final Answer:

   Manually coding software without AI help -> Option A

4. Quick Check:

   GenAI applications exclude manual tasks = A [OK]

Hint: Look for the option that does not involve AI generation [OK]

Common Mistakes:

• Confusing manual tasks as AI applications
• Thinking all coding is GenAI
• Ignoring audio as a GenAI output

2. Which of these is the correct way to prompt a GenAI model to generate an image?

easy

A. Write code to manually draw the image pixel by pixel

B. Upload a photo and ask the model to delete it

C. Type 'Generate a photo of a sunset over mountains' as input

D. Ask the model to write a poem about sunsets

Solution

1. Step 1: Understand how to prompt GenAI for images

   You give a text description like 'Generate a photo of a sunset over mountains' to get an image.

2. Step 2: Identify the correct prompt among options

   Type 'Generate a photo of a sunset over mountains' as input is a clear text prompt for image generation; others are unrelated or incorrect.

3. Final Answer:

   Type 'Generate a photo of a sunset over mountains' as input -> Option C

4. Quick Check:

   Text prompt for image generation = B [OK]

Hint: Choose the option with a clear text description for image generation [OK]

Common Mistakes:

• Confusing manual drawing with AI generation
• Uploading photos is not prompting generation
• Mixing text generation with image generation

3. Given this Python code using a GenAI text model:

prompt = "Write a short poem about spring"
response = genai_model.generate(prompt)
print(response)

What is the most likely output?

medium

A. SyntaxError: invalid syntax

B. "Spring blooms bright, with colors anew, Nature wakes up, fresh morning dew."

C. A blank line with no output

D. An image file of flowers

Solution

1. Step 1: Understand the code's purpose

   The code sends a prompt to a GenAI text model to generate a poem about spring.

2. Step 2: Predict the output type

   The model returns a text poem, so the printed output is a short poem about spring.

3. Final Answer:

   "Spring blooms bright, with colors anew, Nature wakes up, fresh morning dew." -> Option B

4. Quick Check:

   GenAI text generation outputs text poem = A [OK]

Hint: GenAI text prompts return text, not errors or images [OK]

Common Mistakes:

• Expecting code errors from correct syntax
• Confusing text output with image output
• Assuming no output from model call

4. You try to generate audio with this code snippet:

audio = genai_model.generate_audio(prompt="Play a relaxing tune")
print(audio)

But you get an error: AttributeError: 'GenAIModel' object has no attribute 'generate_audio'. What is the likely fix?

medium

A. Use the correct method name, like generate(), for audio generation

B. Change the prompt to text instead of audio

C. Restart the computer to fix the error

D. Remove the print statement

Solution

1. Step 1: Analyze the error message

   The error says the model object has no method named 'generate_audio'.

2. Step 2: Correct the method call

   Use the existing method like 'generate()' that supports audio generation via prompt.

3. Final Answer:

   Use the correct method name, like generate(), for audio generation -> Option A

4. Quick Check:

   Fix method name to existing one = C [OK]

Hint: Check method names carefully in error messages [OK]

Common Mistakes:

• Ignoring error details
• Changing prompt instead of method
• Restarting without debugging code

5. You want to build a GenAI app that takes a user's text prompt and returns both an image and a short audio description. Which approach best combines these tasks?

hard

A. Use one GenAI model that supports multi-modal outputs for text, image, and audio

B. Ask users to upload images and audio instead of generating them

C. Generate only text and convert it manually to image and audio later

D. Use separate GenAI models: one for text-to-image, another for text-to-audio, then combine results

Solution

1. Step 1: Understand multi-modal generation needs

   Generating both image and audio from text usually requires specialized models for each type.

2. Step 2: Choose best practical approach

   Using separate models for text-to-image and text-to-audio then combining outputs is common and effective.

3. Final Answer:

   Use separate GenAI models: one for text-to-image, another for text-to-audio, then combine results -> Option D

4. Quick Check:

   Separate models for different media = D [OK]

Hint: Combine specialized models for different media types [OK]

Common Mistakes:

• Assuming one model handles all media perfectly
• Ignoring need to combine outputs
• Asking users to upload instead of generating