Resume tokens for reliability in MongoDB

Introduction

Resume tokens help you continue reading data from where you left off without missing or repeating anything.

When you want to watch changes in a database and not lose track if your app restarts.

When you read a large list of updates and want to pause and continue later.

When your network connection drops and you want to resume listening without starting over.

When you process data streams and need to avoid duplicates or gaps.

When building reliable real-time apps that react to database changes.

Syntax

MongoDB

const changeStream = collection.watch([], { resumeAfter: resumeToken });

The resumeToken is a special marker from a previous change stream event.

Use resumeAfter option to start watching changes from that marker.

Examples

Start watching changes from now without a resume token.

MongoDB

const changeStream = collection.watch();

Resume watching changes from the last saved token.

MongoDB

const changeStream = collection.watch([], { resumeAfter: lastResumeToken });

Save the resume token from each change event to resume later.

MongoDB

changeStream.on('change', (change) => {
  lastResumeToken = change._id;
});

Sample Program

This program connects to MongoDB, watches changes on a collection, saves the resume token from each change, and stops after 10 seconds.

MongoDB

const { MongoClient } = require('mongodb');

async function watchWithResume() {
  const client = new MongoClient('mongodb://localhost:27017');
  await client.connect();
  const db = client.db('testdb');
  const collection = db.collection('testcol');

  let resumeToken = null;

  // Start watching changes
  const changeStream = collection.watch([], resumeToken ? { resumeAfter: resumeToken } : {});

  changeStream.on('change', (change) => {
    console.log('Change detected:', change);
    resumeToken = change._id; // Save token to resume later
  });

  // Simulate running for 10 seconds
  setTimeout(async () => {
    await changeStream.close();
    await client.close();
    console.log('Stopped watching changes.');
  }, 10000);
}

watchWithResume();

Important Notes

Resume tokens are unique IDs for each change event.

If you lose the token, you must start watching from the beginning or current time.

Always save the token after processing each change to avoid missing data.

Summary

Resume tokens let you continue watching database changes reliably.

Use the resumeAfter option with the saved token to resume.

Save tokens after each event to avoid missing or repeating changes.

Practice

1. What is the main purpose of using resume tokens in MongoDB change streams?

easy

A. To speed up database queries

B. To encrypt the data in the database

C. To backup the entire database

D. To continue watching database changes without missing any events

Solution

1. Step 1: Understand what resume tokens track

   Resume tokens mark the last seen event in a change stream to continue watching from that point.

2. Step 2: Identify the purpose of resume tokens

   They help avoid missing or repeating events when watching database changes.

3. Final Answer:

   To continue watching database changes without missing any events -> Option D

4. Quick Check:

   Resume tokens = Continue watching changes reliably [OK]

Hint: Resume tokens help continue watching changes without gaps [OK]

Common Mistakes:

• Thinking resume tokens speed up queries
• Confusing tokens with backups
• Assuming tokens encrypt data

2. Which option is used to resume a MongoDB change stream from a saved resume token?

easy

A. resumeAfter

B. restartFrom

C. continueFrom

D. startAfter

Solution

1. Step 1: Recall the option name for resuming change streams

   The MongoDB change stream API uses the option named resumeAfter to resume from a token.

2. Step 2: Confirm other options are incorrect

   Options like startAfter, continueFrom, and restartFrom are not valid MongoDB options.

3. Final Answer:

   resumeAfter -> Option A

4. Quick Check:

   Resume option = resumeAfter [OK]

Hint: Use resumeAfter option with saved token to resume [OK]

Common Mistakes:

• Using startAfter instead of resumeAfter
• Confusing option names
• Assuming continueFrom is valid

3. Given the following code snippet, what will be the output if the resume token is correctly used?

const changeStream = collection.watch([], { resumeAfter: savedToken });
changeStream.on('change', (change) => {
  console.log(change._id);
});

medium

A. It will print the _id of changes starting after the savedToken event

B. It will throw a syntax error due to invalid option

C. It will print the _id of all changes from the beginning of the collection

D. It will print no changes because resumeAfter disables events

Solution

1. Step 1: Understand the effect of resumeAfter option

   The resumeAfter option tells MongoDB to start the change stream from the event after the saved token.

2. Step 2: Analyze the output of the event listener

   The listener prints the _id of each change event starting after the saved token.

3. Final Answer:

   It will print the _id of changes starting after the savedToken event -> Option A

4. Quick Check:

   resumeAfter = start after saved token [OK]

Hint: resumeAfter starts stream after saved token event [OK]

Common Mistakes:

• Thinking it prints all changes from start
• Assuming resumeAfter disables events
• Confusing syntax error with option usage

4. You saved a resume token but your change stream still misses some events after resuming. What is the most likely cause?

medium

A. The database does not support change streams

B. You used the resumeAfter option correctly

C. You did not save the resume token after the last event

D. You used the startAfter option instead of resumeAfter

Solution

1. Step 1: Identify the importance of saving resume tokens

   To avoid missing events, you must save the resume token after processing each event.

2. Step 2: Understand the impact of not saving tokens timely

   If you resume from an old token, events between the last processed and saved token will be missed.

3. Final Answer:

   You did not save the resume token after the last event -> Option C

4. Quick Check:

   Save token after each event to avoid missing [OK]

Hint: Always save token after each event to avoid missing changes [OK]

Common Mistakes:

• Assuming resumeAfter usage alone prevents missing events
• Confusing startAfter with resumeAfter
• Blaming database support incorrectly

5. You want to build a reliable system that watches MongoDB changes and never misses or repeats events, even if your app restarts. Which approach is best?

hard

A. Restart the change stream from the beginning every time the app restarts

B. Save the resume token after each change event and use resumeAfter on restart

C. Use a fixed timestamp to start watching changes

D. Ignore resume tokens and rely on polling the database

Solution

1. Step 1: Understand the requirement for reliability

   To never miss or repeat events, the system must remember where it left off in the change stream.

2. Step 2: Identify the best practice for resuming

   Saving the resume token after each event and using resumeAfter on restart ensures continuous, reliable watching.

3. Step 3: Evaluate other options

   Restarting from the beginning wastes resources and may repeat events; fixed timestamps can miss events; polling is inefficient and unreliable.

4. Final Answer:

   Save the resume token after each change event and use resumeAfter on restart -> Option B

5. Quick Check:

   Save token + resumeAfter = reliable watching [OK]

Hint: Save token each event and resumeAfter on restart for reliability [OK]

Common Mistakes:

• Restarting from beginning causes repeats
• Using fixed timestamps misses events
• Ignoring tokens leads to unreliable watching