Queue events and monitoring in NestJS - Step-by-Step Execution

Concept Flow - Queue events and monitoring

Queue Initialized

↓

Job Added to Queue

↓

Event: Job Waiting

→Monitor: Log Waiting

↓

Event: Job Active

→Monitor: Log Active

↓

Event: Job Completed

→Monitor: Log Success

↓

Event: Job Failed

→Monitor: Log Failure

↓

Event: Job Stalled

→Monitor: Log Stall

↓

Repeat for next jobs or Shutdown

This flow shows how a queue processes jobs and emits events that can be monitored for status updates.

Execution Sample

NestJS

const queue = new Queue('myQueue');

queue.on('completed', job => console.log(`Job ${job.id} done`));
queue.add({ task: 'sendEmail' });

This code creates a queue, listens for job completion events, and adds a job to the queue.

Execution Table

Step	Event Triggered	Job ID	Action Taken	Monitor Output
1	Job Added	1	Job added to queue	No output
2	Job Waiting	1	Job is waiting to be processed	Log: Job 1 is waiting
3	Job Active	1	Job started processing	Log: Job 1 is active
4	Job Completed	1	Job finished successfully	Log: Job 1 done
5	Job Added	2	Job added to queue	No output
6	Job Waiting	2	Job is waiting to be processed	Log: Job 2 is waiting
7	Job Active	2	Job started processing	Log: Job 2 is active
8	Job Failed	2	Job failed during processing	Log: Job 2 failed
9	Job Added	3	Job added to queue	No output
10	Job Waiting	3	Job is waiting to be processed	Log: Job 3 is waiting
11	Job Active	3	Job started processing	Log: Job 3 is active
12	Job Stalled	3	Job stalled during processing	Log: Job 3 stalled
13	Queue Shutdown	-	Queue stopped processing	Log: Queue shutdown

💡 Queue shutdown after processing jobs and emitting all events.

Variable Tracker

Variable	Start	After Step 1	After Step 4	After Step 8	After Step 12	Final
queue.jobs	[]	[{id:1, status:'waiting'}]	[{id:1, status:'completed'}]	[{id:1, status:'completed'}, {id:2, status:'failed'}]	[{id:1, status:'completed'}, {id:2, status:'failed'}, {id:3, status:'stalled'}]	[{id:1, status:'completed'}, {id:2, status:'failed'}, {id:3, status:'stalled'}]
job.id	N/A	1	1	2	3	N/A
job.status	N/A	waiting	completed	failed	stalled	N/A

Key Moments - 3 Insights

Why do we see a 'Job Waiting' event before 'Job Active'?

The 'Job Waiting' event signals the job is queued but not yet processed. 'Job Active' means processing started. See execution_table rows 2 and 3.

What happens if a job fails? Does the queue stop?

No, the queue continues processing other jobs even if one fails. See execution_table rows 8 and 9 where job 2 fails but job 3 is still added.

What does the 'Job Stalled' event mean?

'Job Stalled' means the job was active but stopped responding temporarily. It's a warning to monitor. See execution_table row 12.

Visual Quiz - 3 Questions

Look at the execution table, what is the status of job 1 at step 4?

A completed

B active

C waiting

D failed

💡 Hint

Check the 'Event Triggered' and 'Action Taken' columns at step 4.

At which step does job 2 fail?

A Step 6

B Step 7

C Step 8

D Step 9

💡 Hint

Look for 'Job Failed' event in the 'Event Triggered' column.

If we remove the listener for 'Job Stalled', what changes in the monitor output?

A No log for stalled jobs

B No logs at all

C No logs for completed jobs

D Queue stops processing

💡 Hint

Refer to the 'Monitor Output' column for 'Job Stalled' event at step 12.

Concept Snapshot

Queue events in NestJS emit status updates like 'waiting', 'active', 'completed', 'failed', and 'stalled'.
Listeners capture these events to monitor job progress.
Add jobs with queue.add(), listen with queue.on(event).
Monitoring helps track job lifecycle and handle errors.
Queue continues processing despite failures.
Stalled jobs indicate temporary issues needing attention.

Full Transcript

In NestJS, queues emit events during job processing such as when a job is added, waiting, active, completed, failed, or stalled. We create a queue instance and add jobs to it. We listen for events like 'completed' to log or monitor job status. The execution table shows each step where events trigger and what actions happen. Variables track job IDs and statuses changing from waiting to completed or failed. Key moments clarify why waiting comes before active, that failures don't stop the queue, and what stalled means. The visual quiz tests understanding of job statuses at specific steps and effects of removing event listeners. This helps beginners see how queue events flow and how monitoring works in real time.

Practice

1. What is the main purpose of using @OnQueueEvent decorators in NestJS queues?

easy

A. To delete jobs from the queue manually

B. To listen and react to specific queue events like job completion or failure

C. To configure the queue's concurrency settings

D. To create new queues dynamically at runtime

Solution

1. Step 1: Understand the role of @OnQueueEvent

   This decorator is used to listen for events emitted by the queue, such as when a job completes or fails.

2. Step 2: Identify what @OnQueueEvent does not do

   It does not create queues, configure concurrency, or delete jobs manually; it only listens to events.

3. Final Answer:

   To listen and react to specific queue events like job completion or failure -> Option B

4. Quick Check:

   @OnQueueEvent listens to events [OK]

Hint: Remember: @OnQueueEvent listens to queue events only [OK]

Common Mistakes:

• Thinking it creates or deletes queues
• Confusing event listening with queue configuration
• Assuming it manages job concurrency

2. Which of the following is the correct syntax to listen for a 'completed' event in a NestJS queue using @OnQueueEvent?

easy

A. @OnQueueEvent('done')
handleDone(job: Job) {
// handle job completion
}

B. @OnQueueEvent('complete')
handleComplete(job: Job) {
// handle job completion
}

C. @OnQueueEvent('completed')
handleCompleted(job: Job) {
// handle job completion
}

D. @OnQueueEvent('finished')
handleFinished(job: Job) {
// handle job completion
}

Solution

1. Step 1: Recall the exact event name for job completion

   The official event name emitted by NestJS queues for job completion is 'completed'.

2. Step 2: Match the syntax with the correct event name

   The syntax must use @OnQueueEvent('completed') to listen properly.

3. Final Answer:

   @OnQueueEvent('completed')\nhandleCompleted(job: Job) {\n // handle job completion\n} -> Option C

4. Quick Check:

   Event name is 'completed' exactly [OK]

Hint: Use exact event names like 'completed' with @OnQueueEvent [OK]

Common Mistakes:

• Using incorrect event names like 'complete' or 'done'
• Misspelling the decorator syntax
• Not passing the event name as a string

3. Given the following code snippet, what will be logged when a job fails in the queue?

@OnQueueEvent('failed')
handleFailed(job: Job, error: Error) {
  console.log(`Job ${job.id} failed with error: ${error.message}`);
}

medium

A. Job <job_id> failed with error: <error_message>

B. Job failed with error: <error_message>

C. Job <job_id> completed successfully

D. Error occurred but no job info logged

Solution

1. Step 1: Analyze the event and handler parameters

   The handler listens to the 'failed' event and receives the job and error objects.

2. Step 2: Understand the console.log output

   The log prints the job's id and the error's message in the format: "Job failed with error: ".

3. Final Answer:

   Job <job_id> failed with error: <error_message> -> Option A

4. Quick Check:

   Failed event logs job id and error message [OK]

Hint: Failed event handlers get job and error objects [OK]

Common Mistakes:

• Ignoring job id in the log message
• Confusing 'failed' with 'completed' event
• Logging incomplete error information

4. You wrote this event listener but it never triggers:

@OnQueueEvent('completed')
handleCompleted() {
  console.log('Job done');
}

What is the likely problem?

medium

A. The event name 'completed' is incorrect

B. You must call the handler manually after job completion

C. The method name must be 'onCompleted' instead of 'handleCompleted'

D. The handler is missing the job parameter to receive event data

Solution

1. Step 1: Check the event handler signature

   Event handlers for 'completed' receive the job object as a parameter, but this method has none.

2. Step 2: Understand why missing parameters cause no trigger

   Without the expected parameter, NestJS cannot properly call the handler, so it never triggers.

3. Final Answer:

   The handler is missing the job parameter to receive event data -> Option D

4. Quick Check:

   Event handlers need correct parameters [OK]

Hint: Always include expected parameters in event handlers [OK]

Common Mistakes:

• Omitting parameters in event handlers
• Changing event names incorrectly
• Thinking handlers must be called manually

5. You want to monitor a NestJS queue to log when jobs are added, completed, or failed. Which combination of event listeners should you implement to cover these events effectively?

hard

A. Use @OnQueueEvent('waiting'), @OnQueueEvent('completed'), and @OnQueueEvent('failed')

B. Use @OnQueueEvent('added'), @OnQueueEvent('done'), and @OnQueueEvent('error')

C. Use @OnQueueEvent('start'), @OnQueueEvent('finish'), and @OnQueueEvent('fail')

D. Use @OnQueueEvent('enqueue'), @OnQueueEvent('success'), and @OnQueueEvent('failure')

Solution

1. Step 1: Identify correct event names for job lifecycle

   In NestJS queues, 'waiting' is emitted when a job is added, 'completed' when done, and 'failed' on failure.

2. Step 2: Verify the other options' event names

   Other options use incorrect or non-existent event names like 'added', 'done', 'error', 'start', 'finish', 'fail', 'enqueue', 'success', 'failure'.

3. Final Answer:

   Use @OnQueueEvent('waiting'), @OnQueueEvent('completed'), and @OnQueueEvent('failed') -> Option A

4. Quick Check:

   Correct event names: waiting, completed, failed [OK]

Hint: Use official event names: waiting, completed, failed [OK]

Common Mistakes:

• Using wrong event names not recognized by NestJS
• Missing the 'waiting' event for job addition
• Confusing 'failed' with 'fail' or 'error'