Why text generation solves real problems in Prompt Engineering / GenAI - Experiment to Prove It

Experiment - Why text generation solves real problems

Problem:We want to create a model that can generate helpful text responses for customer support questions automatically.

Current Metrics:Training loss: 0.05, Validation loss: 0.20, Training accuracy: 98%, Validation accuracy: 70%

Issue:The model is overfitting: it performs very well on training data but poorly on new, unseen questions.

Your Task

Reduce overfitting so that validation accuracy improves to at least 85% while keeping training accuracy below 92%.

You can only change model architecture and training settings.

Do not add more training data.

Keep the same dataset split.

Hint 1

Hint 2

Hint 3

Solution

Prompt Engineering / GenAI

import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Embedding, LSTM, Dense, Dropout
from tensorflow.keras.callbacks import EarlyStopping

# Sample data placeholders (replace with real data)
X_train, y_train = ...  # tokenized input and target sequences
X_val, y_val = ...

model = Sequential([
    Embedding(input_dim=10000, output_dim=64, input_length=100),
    LSTM(128, return_sequences=True),
    Dropout(0.3),
    LSTM(64),
    Dropout(0.3),
    Dense(10000, activation='softmax')
])

model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=0.001),
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])

early_stop = EarlyStopping(monitor='val_loss', patience=3, restore_best_weights=True)

history = model.fit(X_train, y_train, epochs=20, batch_size=64, validation_data=(X_val, y_val), callbacks=[early_stop])

Added dropout layers after LSTM layers to reduce overfitting.

Reduced learning rate from default to 0.001 for smoother training.

Added early stopping to stop training when validation loss stops improving.

Results Interpretation

Before: Training accuracy 98%, Validation accuracy 70% (overfitting)

After: Training accuracy 90%, Validation accuracy 87% (better generalization)

Adding dropout and early stopping helps the model generalize better to new data by reducing overfitting, which improves real-world usefulness of text generation.

Bonus Experiment

Try using a smaller LSTM size or a simpler model to see if it also reduces overfitting.

💡 Hint

Simpler models often overfit less because they have fewer parameters to memorize training data.

Practice

1. Why is text generation useful in real life?
Text generation helps by:

easy

A. Making computers run faster

B. Replacing all human jobs instantly

C. Only generating random words without meaning

D. Creating written content automatically to save time

Solution

1. Step 1: Understand the purpose of text generation

   Text generation is designed to create written content automatically, which helps save time for people.

2. Step 2: Compare options with real use cases

   Options B, C, and D do not match real benefits: it does not replace all jobs instantly, nor produce meaningless words, nor speed up computers. Only A correctly identifies a benefit.

3. Final Answer:

   Creating written content automatically to save time -> Option D

4. Quick Check:

   Text generation saves time by writing content [OK]

Hint: Focus on time-saving benefits of text generation [OK]

Common Mistakes:

• Thinking text generation replaces all jobs
• Believing it only makes random words
• Confusing text generation with hardware speed

2. Which of these is the correct way to give a prompt to a text generation model?

easy

A. Generate text without any input

B. Provide a clear instruction or starting sentence

C. Use random numbers as input

D. Turn off the model before starting

Solution

1. Step 1: Identify how prompts guide text generation

   Prompts are clear instructions or starting sentences that help the model produce useful text.

2. Step 2: Evaluate each option

   Generate text without any input lacks input, so output is random; C uses irrelevant input; D stops the model. Only A correctly guides the model.

3. Final Answer:

   Provide a clear instruction or starting sentence -> Option B

4. Quick Check:

   Prompt = clear instruction [OK]

Hint: Remember: prompts guide the model's output clearly [OK]

Common Mistakes:

• Trying to generate text without input
• Using unrelated data as prompt
• Turning off the model accidentally

3. What will the text generation model most likely produce if given this prompt?

"Write a short email to thank a friend for their help."

medium

A. "1234567890"

B. "The weather is sunny today."

C. "Dear friend, thanks for your help!"

D. "Error: No input provided"

Solution

1. Step 1: Understand the prompt's instruction

   The prompt asks for a short thank-you email to a friend, so the output should be a polite message expressing thanks.

2. Step 2: Match options to expected output

   "Dear friend, thanks for your help!" matches the prompt well. Options A and B are unrelated text, and D is an error message which is incorrect here.

3. Final Answer:

   "Dear friend, thanks for your help!" -> Option C

4. Quick Check:

   Prompt about thank-you email = polite thank-you text [OK]

Hint: Match prompt meaning to output content [OK]

Common Mistakes:

• Choosing unrelated text outputs
• Confusing error messages with output
• Ignoring prompt instructions

4. A text generation model is given the prompt: "Summarize the story about a cat." but it outputs random numbers instead. What is the likely problem?

medium

A. The prompt was unclear or missing

B. The model is designed only for numbers

C. The model was not trained on text data

D. The model is perfect and no problem exists

Solution

1. Step 1: Analyze the prompt and output mismatch

   The prompt asks for a text summary, but the output is random numbers, which suggests the model did not understand the prompt.

2. Step 2: Identify the cause of wrong output

   Usually, unclear or missing prompts cause irrelevant outputs. Options A and C are unlikely if the model is a text generator. D is incorrect because output is wrong.

3. Final Answer:

   The prompt was unclear or missing -> Option A

4. Quick Check:

   Wrong output = unclear prompt [OK]

Hint: Check if prompt matches expected output type [OK]

Common Mistakes:

• Blaming the model without checking prompt
• Assuming model only works with numbers
• Ignoring mismatch between prompt and output

5. You want to use text generation to create summaries of long articles automatically. Which approach best solves this real problem?

hard

A. Provide the full article as a prompt and ask for a summary

B. Give only the article title and expect a summary

C. Input random sentences unrelated to the article

D. Use text generation to generate random stories instead

Solution

1. Step 1: Understand the goal of summarization

   To summarize an article, the model needs the full content to extract key points and create a summary.

2. Step 2: Evaluate each option's effectiveness

   Provide the full article as a prompt and ask for a summary provides the full article as input, enabling accurate summaries. B lacks content, C is unrelated input, and A does not address summarization.

3. Final Answer:

   Provide the full article as a prompt and ask for a summary -> Option A

4. Quick Check:

   Full input for summary = best results [OK]

Hint: Give full content to summarize, not just title [OK]

Common Mistakes:

• Using incomplete input for summaries
• Expecting summaries from unrelated text
• Confusing story generation with summarization