Metadata filtering in vector stores in LangChain - Mini Project: Build & Apply

Metadata filtering in vector stores

📖 Scenario: You are building a simple document search system using LangChain and a vector store. Your documents have metadata like category and author. You want to filter search results by metadata to find only documents matching certain criteria.

🎯 Goal: Create a LangChain vector store with documents and metadata. Then add a metadata filter to search only documents with category equal to technology.

📋 What You'll Learn

Create a list of documents with text and metadata

Create a LangChain vector store from the documents

Define a metadata filter dictionary with category set to technology

Use the metadata filter in the vector store search method

💡 Why This Matters

🌍 Real World

Filtering search results by metadata is common in document search, recommendation systems, and knowledge bases to improve relevance.

💼 Career

Understanding metadata filtering in vector stores is useful for building intelligent search applications and working with modern AI frameworks like LangChain.

1

Create documents with metadata

Create a list called documents with two dictionaries. The first dictionary should have text set to 'Learn Python programming' and metadata with {'category': 'technology', 'author': 'Alice'}. The second dictionary should have text set to 'Cooking tips for beginners' and metadata with {'category': 'cooking', 'author': 'Bob'}.

LangChain

# Create the list of documents with text and metadata
# Your code here

Hint

Use a list with two dictionaries. Each dictionary has keys text and metadata. Metadata is another dictionary with category and author.

2

Create a LangChain vector store

Import FAISS from langchain.vectorstores. Create a variable called vector_store by calling FAISS.from_texts with the list of document texts extracted from documents and metadata from documents.

LangChain

documents = [
    {'text': 'Learn Python programming', 'metadata': {'category': 'technology', 'author': 'Alice'}},
    {'text': 'Cooking tips for beginners', 'metadata': {'category': 'cooking', 'author': 'Bob'}}
]

# Import FAISS and create vector_store from document texts and metadata
# Your code here

Hint

Extract texts and metadata from documents using list comprehensions. Then call FAISS.from_texts with these lists.

3

Define metadata filter

Create a dictionary called metadata_filter with a key category and value 'technology'.

LangChain

from langchain.vectorstores import FAISS

texts = [doc['text'] for doc in documents]
metadatas = [doc['metadata'] for doc in documents]
vector_store = FAISS.from_texts(texts, metadatas=metadatas)

# Create metadata_filter dictionary with category 'technology'
# Your code here

Hint

Just create a dictionary with the key category and value 'technology'.

4

Search with metadata filter

Call vector_store.similarity_search with query 'Python' and filter=metadata_filter. Assign the result to a variable called results.

LangChain

from langchain.vectorstores import FAISS

texts = [doc['text'] for doc in documents]
metadatas = [doc['metadata'] for doc in documents]
vector_store = FAISS.from_texts(texts, metadatas=metadatas)

metadata_filter = {'category': 'technology'}

# Search vector_store with query 'Python' and filter=metadata_filter
# Your code here

Hint

Call similarity_search on vector_store with the query string and the filter argument set to metadata_filter.

Practice

1. What is the main purpose of metadata filtering in vector stores?

easy

A. To delete irrelevant data from the vector store

B. To increase the size of the vector store

C. To speed up vector calculations without filtering

D. To narrow search results using extra information about data

Solution

1. Step 1: Understand metadata filtering concept

   Metadata filtering uses extra information (metadata) to limit search results to relevant items.

2. Step 2: Identify the main purpose

   The main goal is to narrow down results, not to delete data or speed up calculations without filtering.

3. Final Answer:

   To narrow search results using extra information about data -> Option D

4. Quick Check:

   Metadata filtering = Narrow search results [OK]

Hint: Metadata filters results by extra info, not by size or speed [OK]

Common Mistakes:

• Confusing filtering with deleting data
• Thinking metadata speeds up vector math alone
• Believing filtering increases store size

2. Which of the following is the correct way to apply metadata filtering in a Langchain vector store search?

easy

A. search(query, filter=[('category', 'news')])

B. search(query, filter='category=news')

C. search(query, filter={'category': 'news'})

D. search(query, filter='category:news')

Solution

1. Step 1: Recall correct filter syntax

   Langchain expects a dictionary for metadata filtering, e.g., {'key': 'value'}.

2. Step 2: Match syntax to options

   Only search(query, filter={'category': 'news'}) uses a dictionary correctly; others use strings or lists incorrectly.

3. Final Answer:

   search(query, filter={'category': 'news'}) -> Option C

4. Quick Check:

   Filter syntax = dictionary format [OK]

Hint: Use dictionary for filters: filter={'key': 'value'} [OK]

Common Mistakes:

• Using string instead of dictionary for filter
• Passing list of tuples instead of dict
• Using colon syntax inside string

3. Given this code snippet using Langchain vector store:

results = vector_store.search('climate change', filter={'source': 'news', 'year': 2023})
print(len(results))

If the vector store has 10 documents with source 'news' and year 2023, and 5 with source 'news' but year 2022, what will be printed?

medium

A. 15

B. 10

C. 5

D. 0

Solution

1. Step 1: Understand filter effect

   The filter requires both 'source' = 'news' and 'year' = 2023 to match.

2. Step 2: Count matching documents

   Only 10 documents match both conditions; 5 with year 2022 do not match.

3. Final Answer:

   10 -> Option B

4. Quick Check:

   Filter matches 10 docs with both conditions [OK]

Hint: Filter matches all conditions exactly, not partial [OK]

Common Mistakes:

• Counting documents matching only one filter key
• Ignoring year condition
• Assuming filter returns all 'news' documents

4. You wrote this code but get an error:

results = vector_store.search('AI', filter=['category', 'tech'])

What is the main problem causing the error?

medium

A. Filter should be a dictionary, not a list

B. Search query must be a dictionary

C. Filter keys must be integers

D. Filter cannot be used with string queries

Solution

1. Step 1: Check filter type

   The filter parameter expects a dictionary with key-value pairs.

2. Step 2: Identify incorrect usage

   The code uses a list instead of a dictionary, causing a type error.

3. Final Answer:

   Filter should be a dictionary, not a list -> Option A

4. Quick Check:

   Filter type must be dict, not list [OK]

Hint: Always use dict for filter, not list or other types [OK]

Common Mistakes:

• Passing list instead of dict for filter
• Thinking query must be dict
• Believing filter keys must be numbers

5. You want to find documents in a Langchain vector store about 'machine learning' but only from the year 2022 or later, and with the tag 'tutorial'. Which filter dictionary correctly applies this condition?

hard

A. {'tag': 'tutorial', 'year': {'$gte': 2022}}

B. {'tag': 'tutorial', 'year': 2022}

C. {'tag': ['tutorial'], 'year': 2022}

D. {'tag': 'tutorial', 'year': [2022, 2023, 2024]}

Solution

1. Step 1: Understand filtering for range conditions

   To filter for year 2022 or later, a comparison operator like '$gte' (greater than or equal) is needed.

2. Step 2: Analyze options for correct syntax

   {'tag': 'tutorial', 'year': {'$gte': 2022}} uses {'year': {'$gte': 2022}} which correctly expresses the condition; others do not support range filtering properly.

3. Final Answer:

   {'tag': 'tutorial', 'year': {'$gte': 2022}} -> Option A

4. Quick Check:

   Use comparison operators for range filters [OK]

Hint: Use {'key': {'$gte': value}} for filtering ranges [OK]

Common Mistakes:

• Using exact value instead of range operator
• Passing list instead of operator for year
• Misunderstanding how to filter numeric ranges