Experiment - What Generative AI actually is

Problem:You want to understand how a generative AI model creates new content like text or images based on examples it learned from.

Current Metrics:N/A - This is a conceptual understanding experiment without numeric metrics.

Issue:Many beginners confuse generative AI with simple copying or rule-based systems. They don't see how the model learns patterns and creates new, original outputs.

Your Task

Build a simple generative AI model that learns from a small text dataset and generates new sentences. Understand how the model creates new content rather than copying.

Use a simple character-level RNN model in Python with TensorFlow or PyTorch.

Train on a small sample text (like a few sentences).

Generate new text after training.

Hint 1

Hint 2

Hint 3

Solution

Prompt Engineering / GenAI

import tensorflow as tf
import numpy as np

# Sample text data
text = "hello world. this is generative ai."

# Create character vocabulary
vocab = sorted(set(text))
char2idx = {u:i for i, u in enumerate(vocab)}
idx2char = np.array(vocab)

# Convert text to integer indices
text_as_int = np.array([char2idx[c] for c in text])

# Create input-target sequences
seq_length = 10
examples_per_epoch = len(text)//seq_length

char_dataset = tf.data.Dataset.from_tensor_slices(text_as_int)
sequences = char_dataset.batch(seq_length+1, drop_remainder=True)

# Function to split input and target

def split_input_target(chunk):
    input_text = chunk[:-1]
    target_text = chunk[1:]
    return input_text, target_text

dataset = sequences.map(split_input_target)

# Batch size
BATCH_SIZE = 2
BUFFER_SIZE = 100

dataset = dataset.shuffle(BUFFER_SIZE).batch(BATCH_SIZE, drop_remainder=True)

# Build the model
vocab_size = len(vocab)
embedding_dim = 8
rnn_units = 64

model = tf.keras.Sequential([
    tf.keras.layers.Embedding(vocab_size, embedding_dim, batch_input_shape=[BATCH_SIZE, None]),
    tf.keras.layers.SimpleRNN(rnn_units, return_sequences=True, stateful=True, recurrent_initializer='glorot_uniform'),
    tf.keras.layers.Dense(vocab_size)
])

# Loss function
loss = tf.losses.SparseCategoricalCrossentropy(from_logits=True)
model.compile(optimizer='adam', loss=loss)

# Train the model
EPOCHS = 20
model.fit(dataset, epochs=EPOCHS)

# Text generation function
def generate_text(model, start_string, num_chars=50):
    input_eval = [char2idx[s] for s in start_string]
    input_eval = tf.expand_dims(input_eval, 0)

    text_generated = []

    model.reset_states()
    for _ in range(num_chars):
        predictions = model(input_eval)
        predictions = tf.squeeze(predictions, 0)

        predicted_id = tf.random.categorical(predictions, num_samples=1)[-1,0].numpy()

        input_eval = tf.expand_dims([predicted_id], 0)

        text_generated.append(idx2char[predicted_id])

    return start_string + ''.join(text_generated)

# Generate new text
print(generate_text(model, start_string="hello "))

Built a simple character-level RNN model to learn patterns in text.

Used a small dataset and limited epochs to avoid memorization.

Implemented text generation by predicting one character at a time.

Results Interpretation

Before: No model, no generated text.

After: Model generates new text sequences that resemble the style and characters of the training text but are not exact copies.

Generative AI learns patterns from data and creates new, original content by predicting what comes next, not by copying. This experiment shows how a simple model can generate text character by character.

Bonus Experiment

Try training the model on a larger text dataset and increase the number of epochs. Observe how the generated text improves or starts to copy exact phrases.

💡 Hint

Watch for overfitting: if the model memorizes, generated text will repeat training sentences exactly. Use dropout or reduce epochs to keep creativity.