Inpainting and outpainting in Prompt Engineering / GenAI - Model Pipeline Trace

Model Pipeline - Inpainting and outpainting

This pipeline shows how a generative AI model fills missing parts inside an image (inpainting) or extends the image beyond its original borders (outpainting). It learns from existing image data to create realistic new pixels.

Data Flow - 5 Stages

1Input Image

1 image x 256 x 256 x 3 channels→Original image with missing or extended areas marked→1 image x 256 x 256 x 3 channels

A photo of a landscape with a blank square in the center (inpainting) or blank space on the right edge (outpainting)

↓

2Mask Application

1 image x 256 x 256 x 3 channels→Apply mask to indicate missing or new areas to fill→1 image x 256 x 256 x 4 channels (RGB + mask)

The image now includes a mask channel showing which pixels to generate

↓

3Feature Extraction

1 image x 256 x 256 x 4 channels→Use convolutional layers to extract image features→1 tensor x 64 x 64 x 256 features

Features capture edges, colors, and textures from the visible parts

↓

4Generative Model

1 tensor x 64 x 64 x 256 features→Use a U-Net or transformer-based model to generate missing pixels→1 image x 256 x 256 x 3 channels

Model predicts realistic pixels for masked or extended areas

↓

5Output Image

1 image x 256 x 256 x 3 channels→Combine generated pixels with original image→1 image x 256 x 256 x 3 channels

Completed image with missing parts filled or extended beyond original edges

Training Trace - Epoch by Epoch

Loss
1.2 |*****
1.0 |****
0.8 |***
0.6 |**
0.4 |*
0.2 | 
    +----------------
     1 5 10 15 20 Epochs

Epoch	Loss ↓	Accuracy ↑	Observation
1	1.2	0.45	Model starts learning to fill missing areas roughly
5	0.8	0.65	Generated pixels become more coherent with surroundings
10	0.5	0.80	Model produces realistic textures and colors
15	0.35	0.88	Fine details and edges are well reconstructed
20	0.25	0.92	Model converges with high-quality inpainting and outpainting

Prediction Trace - 4 Layers

Layer 1: Input Image with Mask

Layer 2: Feature Extraction

Layer 3: Generative Model (U-Net or Transformer)

Layer 4: Combine Output

Model Quiz - 3 Questions

What does the mask channel tell the model during inpainting?

A Which pixels to generate or fill

B The color of the missing pixels

C The size of the image

D The brightness of the image

Key Insight

Inpainting and outpainting models learn to fill or extend images by understanding visible patterns and textures. Masks guide where to generate pixels, and training improves the model's ability to create realistic, seamless images.