Contextual compression in LangChain - Step-by-Step Execution

Concept Flow - Contextual compression

Input: Long Text

↓

Split Text into Chunks

↓

Compress Each Chunk Using Context

↓

Combine Compressed Chunks

↓

Output: Compressed Text

The process breaks a long text into smaller parts, compresses each part using context, then combines them into a shorter version.

Execution Sample

LangChain

from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.text_compression import ContextualCompression

text = "This is a long text that needs compression."
splitter = RecursiveCharacterTextSplitter(chunk_size=20)
chunks = splitter.split_text(text)
compressor = ContextualCompression()
compressed = [compressor.compress(chunk) for chunk in chunks]

This code splits a long text into chunks and compresses each chunk contextually.

Execution Table

Step	Action	Input	Output	Notes
1	Split text	"This is a long text that needs compression."	["This is a long text ", "that needs compression."]	Text split into two chunks of max 20 chars
2	Compress chunk 1	"This is a long text "	"This is long text"	Compression removes redundant words using context
3	Compress chunk 2	"that needs compression."	"needs compression"	Compression keeps key meaning
4	Combine compressed chunks	["This is long text", "needs compression"]	"This is long text needs compression"	Chunks joined into compressed text

💡 All chunks compressed and combined into final compressed text

Variable Tracker

Variable	Start	After Step 1	After Step 2	After Step 3	After Step 4
text	"This is a long text that needs compression."	"This is a long text that needs compression."	"This is a long text that needs compression."	"This is a long text that needs compression."	"This is a long text that needs compression."
chunks	[]	["This is a long text ", "that needs compression."]	["This is a long text ", "that needs compression."]	["This is a long text ", "that needs compression."]	["This is a long text ", "that needs compression."]
compressed	[]	[]	["This is long text"]	["This is long text", "needs compression"]	["This is long text", "needs compression"]
final_text	""	""	""	""	"This is long text needs compression"

Key Moments - 3 Insights

Why do we split the text before compressing?

Splitting breaks the long text into smaller parts that are easier to compress effectively, as shown in execution_table step 1.

Does compression remove all words?

No, compression removes only redundant or less important words while keeping the main meaning, as seen in steps 2 and 3.

How do compressed chunks become the final text?

Compressed chunks are combined in order to form a shorter but meaningful text, as shown in step 4.

Visual Quiz - 3 Questions

Look at the execution_table, what is the output after compressing the first chunk?

A "needs compression"

B "This is a long text "

C "This is long text"

D "This is a long text that needs compression."

💡 Hint

Check execution_table row 2 under Output column

At which step are the chunks combined into the final compressed text?

A Step 2

B Step 4

C Step 3

D Step 1

💡 Hint

Look at execution_table rows and find where 'Combine compressed chunks' happens

If the chunk size was larger, how would the number of chunks change?

A Fewer chunks

B More chunks

C Same number of chunks

D Chunks would be empty

💡 Hint

Refer to variable_tracker 'chunks' after Step 1 and think about chunk size effect

Concept Snapshot

Contextual compression breaks long text into smaller chunks.
Each chunk is compressed by removing redundant words using context.
Compressed chunks are combined to form a shorter meaningful text.
Use text splitters and compressors from Langchain.
This helps reduce text size while keeping key information.

Full Transcript

Contextual compression in Langchain works by first splitting a long text into smaller chunks. Each chunk is then compressed by removing unnecessary words while keeping the main meaning. Finally, the compressed chunks are combined to produce a shorter version of the original text. This process helps manage large texts efficiently by reducing their size but preserving important content. The example code shows splitting a text into two parts and compressing each before joining them back. The execution table traces each step, showing inputs and outputs. Variables like 'chunks' and 'compressed' track the state changes. Key moments clarify why splitting is needed and how compression keeps meaning. The quiz tests understanding of outputs at each step and effects of chunk size. Overall, contextual compression is a useful technique to shorten text smartly using Langchain tools.