Metadata preservation during splitting in LangChain - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ component_behavior

intermediate

What happens to metadata after splitting?

Given a document with metadata, what will be the metadata of the first chunk after splitting using a typical LangChain text splitter?

LangChain

from langchain.schema import Document
from langchain.text_splitter import RecursiveCharacterTextSplitter

doc = Document(page_content="Hello world! This is a test.", metadata={"source": "test_source", "page": 1})
splitter = RecursiveCharacterTextSplitter(chunk_size=12, chunk_overlap=0)
chunks = splitter.split_documents([doc])
first_chunk_metadata = chunks[0].metadata

A {"source": "test_source", "page": 1}

B {}

C {"source": "test_source"}

D None

💡 Hint

Think about whether the splitter copies metadata or discards it.

LangChain's RecursiveCharacterTextSplitter preserves the metadata of the original document in each chunk it creates. So each chunk retains the same metadata dictionary.

❓ state_output

intermediate

How many chunks retain metadata after splitting?

If you split a single document with metadata into 3 chunks using LangChain's RecursiveCharacterTextSplitter, how many chunks will have the original metadata preserved?

LangChain

from langchain.schema import Document
from langchain.text_splitter import RecursiveCharacterTextSplitter

doc = Document(page_content="A long text that will be split into three parts.", metadata={"author": "Alice"})
splitter = RecursiveCharacterTextSplitter(chunk_size=20, chunk_overlap=0)
chunks = splitter.split_documents([doc])
count_with_metadata = sum(1 for c in chunks if c.metadata == {"author": "Alice"})

A 1

B 3

C 0

D Depends on chunk content

💡 Hint

Consider if metadata is copied or removed during splitting.

Each chunk created by the splitter retains the original document's metadata, so all 3 chunks have the metadata.

📝 Syntax

advanced

Which code correctly preserves metadata during splitting?

Which of the following code snippets correctly preserves the metadata of documents after splitting using LangChain's RecursiveCharacterTextSplitter?

A chunks = splitter.split_documents([doc])

B chunks = splitter.split_text(doc.page_content)

C chunks = [Document(page_content=chunk) for chunk in splitter.split_text(doc.page_content)]

D chunks = [Document(page_content=chunk, metadata=doc.metadata) for chunk in splitter.split_text(doc.page_content)]

💡 Hint

Look for the method that returns Document objects with metadata.

split_documents returns Document objects preserving metadata. split_text returns strings without metadata, so manual wrapping is needed to preserve metadata.

🔧 Debug

advanced

Why is metadata lost after splitting?

A developer uses splitter.split_text(doc.page_content) and then creates new Document objects from the chunks but forgets to assign metadata. What is the result for metadata in the chunks?

LangChain

chunks_text = splitter.split_text(doc.page_content)
chunks = [Document(page_content=chunk) for chunk in chunks_text]
metadata_list = [chunk.metadata for chunk in chunks]

A None # metadata attribute missing

B [{"source": "original"}, ...] # original metadata preserved

C [{}, {}, ...] # empty metadata dictionaries

D Error: Document constructor requires metadata

💡 Hint

Check what happens if metadata is not passed when creating Document objects.

If metadata is not passed, Document objects have empty metadata dictionaries by default.

🧠 Conceptual

expert

How to ensure custom metadata updates during splitting?

You want to split documents but also add a new metadata field 'chunk_index' to each chunk indicating its order. Which approach correctly achieves this while preserving original metadata?

LangChain

from langchain.schema import Document
from langchain.text_splitter import RecursiveCharacterTextSplitter

doc = Document(page_content="Some long text here.", metadata={"source": "book"})
splitter = RecursiveCharacterTextSplitter(chunk_size=10, chunk_overlap=0)
chunks = splitter.split_documents([doc])
# Add chunk_index metadata here

A chunks = splitter.split_text(doc.page_content); for i, text in enumerate(chunks): chunks[i] = Document(page_content=text, metadata={'chunk_index': i})

B for i, c in enumerate(chunks): c.metadata['chunk_index'] = i

C chunks = [Document(page_content=c.page_content, metadata={'chunk_index': i}) for i, c in enumerate(chunks)]

D chunks = [Document(page_content=c.page_content, metadata={**c.metadata, 'chunk_index': i}) for i, c in enumerate(chunks)]

💡 Hint

Think about preserving original metadata and adding new keys immutably.

Option D creates new Document objects merging original metadata with the new 'chunk_index' key, preserving all original metadata fields.

Practice

1. What is the main benefit of preserving metadata when splitting documents in Langchain?

easy

A. It speeds up the splitting process.

B. It removes unnecessary text from documents.

C. It reduces the size of each chunk.

D. It keeps important context linked to each chunk.

Solution

1. Step 1: Understand what metadata means in this context

   Metadata is extra information about the document, like source or tags, that helps keep context.

2. Step 2: Recognize why preserving metadata is useful

   When splitting, keeping metadata ensures each chunk still knows where it came from and what it relates to.

3. Final Answer:

   It keeps important context linked to each chunk. -> Option D

4. Quick Check:

   Metadata preservation = keeps context [OK]

Hint: Metadata keeps context linked to chunks [OK]

Common Mistakes:

• Thinking metadata speeds splitting
• Confusing metadata with chunk size
• Believing metadata removes text

2. Which method in Langchain is used to split documents while preserving metadata?

easy

A. split_documents

B. preserve_metadata

C. chunk_text

D. split_text

Solution

1. Step 1: Recall the method name for splitting with metadata

   The method designed to split documents and keep metadata is called split_documents.

2. Step 2: Eliminate incorrect options

   Other options like preserve_metadata, chunk_text, and split_text are not valid Langchain methods for this purpose.

3. Final Answer:

   split_documents -> Option A

4. Quick Check:

   Method to split with metadata = split_documents [OK]

Hint: Remember method name: split_documents [OK]

Common Mistakes:

• Using split_text instead of split_documents
• Assuming preserve_metadata is a method
• Confusing chunk_text with splitting

3. Given this code snippet, what will be the metadata of the first chunk after splitting?

docs = [Document(page_content='Hello world', metadata={'source': 'file1'})]
splitter = RecursiveCharacterTextSplitter(chunk_size=5, chunk_overlap=0)
chunks = splitter.split_documents(docs)
print(chunks[0].metadata)

medium

A. null

B. {}

C. {'source': 'file1'}

D. {'chunk': 0}

Solution

1. Step 1: Understand the input document metadata

   The original document has metadata {'source': 'file1'}.

2. Step 2: Check if splitting preserves metadata

   The split_documents method keeps metadata for each chunk, so the first chunk keeps {'source': 'file1'}.

3. Final Answer:

   {'source': 'file1'} -> Option C

4. Quick Check:

   Metadata preserved = {'source': 'file1'} [OK]

Hint: Chunks keep original metadata unchanged [OK]

Common Mistakes:

• Expecting empty metadata after split
• Assuming metadata is lost or null
• Thinking chunk index is added to metadata

4. What is wrong with this code if the metadata is missing after splitting?

docs = [Document(page_content='Test text', metadata={'id': 123})]
splitter = RecursiveCharacterTextSplitter(chunk_size=4, chunk_overlap=1)
chunks = splitter.split_texts(docs)
print(chunks[0].metadata)

medium

A. Using split_texts instead of split_documents loses metadata.

B. Chunk size is too small to keep metadata.

C. Metadata key 'id' is invalid and removed.

D. Overlap must be zero to preserve metadata.

Solution

1. Step 1: Identify the method used for splitting

   The code uses split_texts which only splits text strings, not Document objects with metadata.

2. Step 2: Understand consequence on metadata

   Since split_texts works on strings, metadata is lost. The correct method to preserve metadata is split_documents.

3. Final Answer:

   Using split_texts instead of split_documents loses metadata. -> Option A

4. Quick Check:

   split_texts loses metadata [OK]

Hint: Use split_documents to keep metadata [OK]

Common Mistakes:

• Thinking chunk size affects metadata
• Believing metadata keys cause loss
• Assuming overlap affects metadata

5. You want to split a document into chunks of 100 characters with 20 characters overlapping, preserving metadata. Which code correctly achieves this?

hard

A. splitter = RecursiveCharacterTextSplitter(chunk_size=100, chunk_overlap=20)
chunks = splitter.split_texts(docs)

B. splitter = RecursiveCharacterTextSplitter(chunk_size=100, chunk_overlap=20)
chunks = splitter.split_documents(docs)

C. splitter = RecursiveCharacterTextSplitter(chunk_size=80, chunk_overlap=20)
chunks = splitter.split_documents(docs)

D. splitter = RecursiveCharacterTextSplitter(chunk_size=100, chunk_overlap=0)
chunks = splitter.split_documents(docs)

Solution

1. Step 1: Set chunk size and overlap correctly

   The question asks for 100 characters chunk size and 20 characters overlap, so chunk_size=100 and chunk_overlap=20.

2. Step 2: Use the correct splitting method to preserve metadata

   To keep metadata, use split_documents method, not split_texts.

3. Final Answer:

   splitter = RecursiveCharacterTextSplitter(chunk_size=100, chunk_overlap=20) chunks = splitter.split_documents(docs) -> Option B

4. Quick Check:

   Correct chunk size, overlap, and method = splitter = RecursiveCharacterTextSplitter(chunk_size=100, chunk_overlap=20) chunks = splitter.split_documents(docs) [OK]

Hint: Match chunk size, overlap, and use split_documents [OK]

Common Mistakes:

• Using split_texts instead of split_documents
• Setting wrong chunk size or overlap
• Using zero overlap when asked for 20