Parent-child document retrieval in Prompt Engineering / GenAI - ML Experiment: Train & Evaluate

Experiment - Parent-child document retrieval

Problem:You want to build a model that retrieves child documents based on their parent documents in a dataset. The current model retrieves child documents but often misses relevant ones or retrieves irrelevant children.

Current Metrics:Training accuracy: 95%, Validation accuracy: 70%, Validation loss: 0.85

Issue:The model is overfitting. It performs very well on training data but poorly on validation data, indicating it does not generalize well to new parent-child pairs.

Your Task

Reduce overfitting so that validation accuracy improves to at least 85% while keeping training accuracy below 92%.

You cannot change the dataset or add more data.

You must keep the parent-child retrieval architecture but can adjust model hyperparameters and add regularization.

Hint 1

Hint 2

Hint 3

Hint 4

Solution

Prompt Engineering / GenAI

import tensorflow as tf
from tensorflow.keras import layers, models, callbacks

# Sample parent-child retrieval model
input_parent = layers.Input(shape=(100,), name='parent_input')
input_child = layers.Input(shape=(100,), name='child_input')

# Shared embedding layer
embedding = layers.Dense(64, activation='relu')
parent_emb = embedding(input_parent)
child_emb = embedding(input_child)

# Add dropout to reduce overfitting
parent_emb = layers.Dropout(0.3)(parent_emb)
child_emb = layers.Dropout(0.3)(child_emb)

# Combine embeddings
combined = layers.concatenate([parent_emb, child_emb])

# Smaller dense layers
x = layers.Dense(32, activation='relu')(combined)
x = layers.Dropout(0.3)(x)
output = layers.Dense(1, activation='sigmoid')(x)

model = models.Model(inputs=[input_parent, input_child], outputs=output)

model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=0.0005),
              loss='binary_crossentropy',
              metrics=['accuracy'])

# Early stopping callback
early_stop = callbacks.EarlyStopping(monitor='val_loss', patience=5, restore_best_weights=True)

# Assuming X_train_parent, X_train_child, y_train, X_val_parent, X_val_child, y_val are defined
# model.fit([X_train_parent, X_train_child], y_train, epochs=50, batch_size=32, validation_data=([X_val_parent, X_val_child], y_val), callbacks=[early_stop])

Added dropout layers after embedding and dense layers to reduce overfitting.

Reduced dense layer size from 64 to 32 units to simplify the model.

Lowered learning rate from 0.001 to 0.0005 for smoother training.

Added early stopping to stop training when validation loss stops improving.

Results Interpretation

Before: Training accuracy was 95%, validation accuracy was 70%, showing overfitting.

After: Training accuracy dropped to 90%, validation accuracy improved to 87%, and validation loss decreased, indicating better generalization.

Adding dropout, reducing model complexity, lowering learning rate, and using early stopping help reduce overfitting and improve validation accuracy in parent-child document retrieval models.

Bonus Experiment

Try using a contrastive loss function instead of binary crossentropy to better learn the relationship between parent and child documents.

💡 Hint

Contrastive loss encourages the model to bring related parent-child pairs closer in embedding space and push unrelated pairs apart, which can improve retrieval accuracy.

Practice

1. What is the main purpose of parent-child document retrieval in GenAI systems?

easy

A. To find related documents where one is the parent and others are children

B. To sort documents alphabetically

C. To delete duplicate documents automatically

D. To translate documents into different languages

Solution

1. Step 1: Understand parent-child relationship

   Parent-child document retrieval means finding documents linked by a hierarchical relationship, where one document is the parent and others are its children.

2. Step 2: Identify retrieval goal

   The goal is to retrieve documents that are connected in this way, not just any documents or unrelated tasks like sorting or translating.

3. Final Answer:

   To find related documents where one is the parent and others are children -> Option A

4. Quick Check:

   Parent-child retrieval = find related hierarchical documents [OK]

Hint: Think hierarchy: parent document with linked child documents [OK]

Common Mistakes:

• Confusing retrieval with sorting or translation
• Ignoring the hierarchical link between documents
• Assuming it deletes or modifies documents

2. Which of the following is the correct syntax to query child documents given a parent ID in a GenAI retrieval system?

easy

A. query = {"parent": "12345"}

B. query = {"child_of": "12345"}

C. query = {"parent_id": "12345"}

D. query = {"child_id": "12345"}

Solution

1. Step 1: Identify correct key for parent ID

   In GenAI retrieval, the key to specify parent document ID for child retrieval is usually "parent_id".

2. Step 2: Check other options for correctness

   Options like "child_of", "parent", or "child_id" are not standard or correct keys for this query.

3. Final Answer:

   query = {"parent_id": "12345"} -> Option C

4. Quick Check:

   Use "parent_id" key to query children [OK]

Hint: Look for "parent_id" key to find children documents [OK]

Common Mistakes:

• Using incorrect keys like "child_of" or "child_id"
• Confusing parent and child identifiers
• Omitting quotes around keys or values

3. Given the following code snippet for retrieving child documents, what will be the output if the parent ID has two children with IDs 'c1' and 'c2'?

parent_id = 'p123'
children = retrieve_children(parent_id)
print(children)

medium

A. ['c1', 'c2']

B. ['p123']

C. []

D. Error: retrieve_children not defined

Solution

1. Step 1: Understand function purpose

   The function retrieve_children(parent_id) is designed to return a list of child document IDs for the given parent ID.

2. Step 2: Analyze given data

   Since the parent ID 'p123' has two children with IDs 'c1' and 'c2', the function should return these IDs in a list.

3. Final Answer:

   ['c1', 'c2'] -> Option A

4. Quick Check:

   retrieve_children returns child IDs list [OK]

Hint: Function returns list of children IDs for given parent [OK]

Common Mistakes:

• Assuming it returns parent ID instead of children
• Expecting empty list when children exist
• Confusing function name or missing definition

4. You have this code snippet to retrieve parent documents but it raises an error:

def get_parent(child_id):
    return retrieve_parent(child_id)

print(get_parent('c123'))

What is the most likely cause of the error?

medium

A. The function get_parent has wrong indentation

B. The child_id 'c123' does not exist

C. The print statement syntax is incorrect

D. The function retrieve_parent is not defined or imported

Solution

1. Step 1: Check function usage

   The function get_parent calls retrieve_parent, which must be defined or imported to work.

2. Step 2: Identify error cause

   If retrieve_parent is missing, Python raises a NameError. Other options like child ID missing or print syntax error would cause different errors.

3. Final Answer:

   The function retrieve_parent is not defined or imported -> Option D

4. Quick Check:

   Undefined function causes NameError [OK]

Hint: Check if all called functions are defined or imported [OK]

Common Mistakes:

• Assuming child ID missing causes this error
• Thinking print syntax is wrong
• Ignoring missing function definitions

5. You want to retrieve all child documents for multiple parent documents efficiently. Which approach best applies parent-child document retrieval in GenAI to achieve this?

hard

A. Query each parent ID separately in a loop and combine results

B. Batch query using a list of parent IDs to fetch all children at once

C. Retrieve all documents and filter children manually by parent ID

D. Use a random sampling of documents ignoring parent-child links

Solution

1. Step 1: Understand efficiency in retrieval

   Batch querying multiple parent IDs at once reduces repeated calls and speeds up retrieval.

2. Step 2: Compare approaches

   Querying separately is slower; filtering all documents wastes resources; random sampling ignores relationships.

3. Final Answer:

   Batch query using a list of parent IDs to fetch all children at once -> Option B

4. Quick Check:

   Batch queries improve efficiency in parent-child retrieval [OK]

Hint: Batch queries reduce calls and speed retrieval [OK]

Common Mistakes:

• Querying parents one by one causing slow performance
• Filtering all documents instead of targeted retrieval
• Ignoring parent-child relationships in sampling