Change stream pipelines for filtering in MongoDB - Step-by-Step Execution

Concept Flow - Change stream pipelines for filtering

Start watching collection

↓

Receive change event

↓

Apply pipeline filter

↓

Emit event

↓

Wait for next event

This flow shows how MongoDB change streams receive events, apply filters using a pipeline, and emit only matching events.

Execution Sample

MongoDB

db.collection.watch([
  { $match: { 'fullDocument.status': 'active' } }
])

This pipeline watches a collection and only emits change events where the document's status is 'active'.

Execution Table

Step	Change Event Received	Pipeline Filter Condition	Filter Result	Action
1	{_id:1, fullDocument:{status:'active'}}	status == 'active'	True	Emit event
2	{_id:2, fullDocument:{status:'inactive'}}	status == 'active'	False	Ignore event
3	{_id:3, fullDocument:{status:'active'}}	status == 'active'	True	Emit event
4	{_id:4, fullDocument:{status:'pending'}}	status == 'active'	False	Ignore event
5	No more events	-	-	Stop watching

💡 No more change events to process, watching ends.

Variable Tracker

Variable	Start	After 1	After 2	After 3	After 4	Final
Current Event	null	{_id:1, status:'active'}	{_id:2, status:'inactive'}	{_id:3, status:'active'}	{_id:4, status:'pending'}	null
Filter Result	null	True	False	True	False	null
Emitted Events Count	0	1	1	2	2	2

Key Moments - 2 Insights

Why does the event with status 'inactive' get ignored?

Because the pipeline filter only allows events where status is 'active' (see execution_table row 2), so 'inactive' does not pass the filter.

What happens if no events match the filter?

No events are emitted; the change stream waits for new events but does not output anything until a matching event arrives (see execution_table rows 2 and 4).

Visual Quiz - 3 Questions

Look at the execution table, how many events were emitted after step 3?

A 3

B 2

C 1

D 0

💡 Hint

Check the 'Emitted Events Count' column in variable_tracker after step 3.

At which step does the filter condition evaluate to False for the first time?

A Step 1

B Step 3

C Step 2

D Step 4

💡 Hint

Look at the 'Filter Result' column in execution_table for each step.

If the pipeline filter changed to status == 'pending', how many events would be emitted by step 4?

A 1

B 2

C 3

D 0

💡 Hint

Check which events have status 'pending' in the 'Change Event Received' column.

Concept Snapshot

Change stream pipelines filter events in MongoDB.
Use $match stage to specify conditions.
Only events matching the filter are emitted.
Non-matching events are ignored silently.
Useful to watch specific changes efficiently.

Full Transcript

This visual execution shows how MongoDB change streams use pipelines to filter events. The stream receives change events from a collection. Each event is checked against the pipeline filter condition, for example, matching documents with status 'active'. If the event matches, it is emitted to the client. If not, it is ignored. The process repeats for each event until no more events arrive. Variables like current event, filter result, and emitted events count change step by step. This helps beginners see exactly how filtering works in change streams.

Practice

1. What is the main purpose of using a $match stage in a MongoDB change stream pipeline?

easy

A. To insert new documents into the collection

B. To create indexes on the collection

C. To delete documents from the collection

D. To filter change events and receive only relevant updates

Solution

1. Step 1: Understand change streams

   Change streams allow applications to listen to changes in MongoDB collections or databases.

2. Step 2: Role of $match in pipelines

   The $match stage filters these change events so only specific changes are passed through.

3. Final Answer:

   To filter change events and receive only relevant updates -> Option D

4. Quick Check:

   $match filters events [OK]

Hint: Remember: $match filters events to reduce noise [OK]

Common Mistakes:

• Thinking $match modifies data instead of filtering
• Confusing $match with data insertion or deletion
• Assuming $match creates indexes

2. Which of the following is the correct syntax to filter change stream events for only insert operations using $match?

easy

A. { $match: { operationType: 'insert' } }

B. { $match: { operationType: 'replace' } }

C. { $match: { operationType: 'delete' } }

D. { $match: { operationType: 'update' } }

Solution

1. Step 1: Identify operationType for inserts

   MongoDB change streams use operationType to specify the type of change, and 'insert' is the value for insert operations.

2. Step 2: Match only insert events

   The correct filter uses { $match: { operationType: 'insert' } } to watch only insert changes.

3. Final Answer:

   { $match: { operationType: 'insert' } } -> Option A

4. Quick Check:

   Filter inserts with operationType 'insert' [OK]

Hint: Match operationType: 'insert' to filter inserts [OK]

Common Mistakes:

• Using wrong operationType values like 'update' or 'delete'
• Missing quotes around 'insert'
• Using $match incorrectly outside pipeline

3. Given the following change stream pipeline, what events will be received?

[ { $match: { 'fullDocument.status': 'active' } } ]

medium

A. All change events regardless of document status

B. Only delete events

C. Only change events where the updated document has status 'active'

D. Only insert events with status 'inactive'

Solution

1. Step 1: Understand the filter condition

   The pipeline filters events where the fullDocument.status field equals 'active'.

2. Step 2: Effect on change events

   This means only events where the changed document's status is 'active' will pass through.

3. Final Answer:

   Only change events where the updated document has status 'active' -> Option C

4. Quick Check:

   Filter by fullDocument.status = 'active' [OK]

Hint: Filter on fullDocument fields to get specific document changes [OK]

Common Mistakes:

• Assuming all events pass through without filtering
• Confusing delete events which don't have fullDocument
• Thinking it filters by operationType

4. You wrote this change stream pipeline to watch only delete events:

[ { $match: { operationType: 'delete' } } ]

But you still receive update events. What is the likely mistake?

medium

A. The field name should be operation_type instead of operationType

B. The pipeline is not applied correctly or the driver ignores the filter

C. The $match stage is missing in the pipeline

D. Delete events do not have an operationType field

Solution

1. Step 1: Verify field name and filter correctness

   The field operationType is correct and the filter matches delete events properly.

2. Step 2: Consider pipeline application

   If update events still appear, the pipeline might not be applied correctly or the driver ignores the filter.

3. Final Answer:

   The pipeline is not applied correctly or the driver ignores the filter -> Option B

4. Quick Check:

   Incorrect pipeline application causes wrong events [OK]

Hint: Check if pipeline is properly passed to watch() method [OK]

Common Mistakes:

• Using wrong field name like operation_type
• Assuming delete events lack operationType
• Not passing pipeline to watch() function

5. You want to watch changes only for documents where category is 'books' and the operation is either 'insert' or 'update'. Which pipeline correctly filters these events?

hard

A. [ { $match: { $and: [ { 'fullDocument.category': 'books' }, { operationType: { $in: ['insert', 'update'] } } ] } } ]

B. [ { $match: { 'fullDocument.category': 'books', operationType: 'insert' } }, { $match: { operationType: 'update' } } ]

C. [ { $match: { operationType: 'insert' } }, { $match: { 'fullDocument.category': 'books' } } ]

D. [ { $match: { $or: [ { 'fullDocument.category': 'books' }, { operationType: 'insert' }, { operationType: 'update' } ] } } ]

Solution

1. Step 1: Combine conditions with $and

   We want documents where category is 'books' AND operationType is 'insert' or 'update'.

2. Step 2: Use $in for multiple operationTypes

   The correct way is to use $in inside $and to filter operationType for both 'insert' and 'update'.

3. Final Answer:

   [ { $match: { $and: [ { 'fullDocument.category': 'books' }, { operationType: { $in: ['insert', 'update'] } } ] } } ] -> Option A

4. Quick Check:

   Use $and with $in to combine filters [OK]

Hint: Use $and with $in to filter multiple conditions together [OK]

Common Mistakes:

• Using multiple $match stages that override each other
• Using $or instead of $and for combined conditions
• Not using $in for multiple operationTypes