Queue events and monitoring in NestJS

Introduction

Queue events and monitoring help you watch what happens inside your task queues. This lets you know if tasks are waiting, running, or done, so you can fix problems fast.

You want to see if background jobs are working correctly in your NestJS app.

You need to know when a task fails so you can retry or alert someone.

You want to track how many jobs are waiting or completed to improve performance.

You want to log queue activity for debugging or auditing.

You want to build a dashboard showing queue status in real time.

Syntax

NestJS

import { Processor, Process, OnQueueEvent } from '@nestjs/bull';
import { Job } from 'bullmq';

@Processor('myQueue')
export class MyQueueProcessor {
  @Process()
  async handleJob(job: Job) {
    // job processing code here
  }

  @OnQueueEvent('completed')
  onJobCompleted(event: any) {
    console.log('Job completed:', event.jobId);
  }

  @OnQueueEvent('failed')
  onJobFailed(event: any) {
    console.log('Job failed:', event.jobId, 'Reason:', event.failedReason);
  }
}

Use @Processor('queueName') to define which queue to listen to.

@OnQueueEvent('eventName') listens to specific queue events like 'completed', 'failed', 'waiting', etc.

Examples

Basic example listening to 'completed' event for an email queue.

NestJS

import { Processor, Process, OnQueueEvent } from '@nestjs/bull';
import { Job } from 'bullmq';

@Processor('emailQueue')
export class EmailProcessor {
  @Process()
  async sendEmail(job: Job) {
    // send email logic
  }

  @OnQueueEvent('completed')
  onCompleted(event: any) {
    console.log(`Email job ${event.jobId} completed`);
  }
}

Example showing how to catch failed jobs and log the reason.

NestJS

import { Processor, Process, OnQueueEvent } from '@nestjs/bull';
import { Job } from 'bullmq';

@Processor('taskQueue')
export class TaskProcessor {
  @Process()
  async runTask(job: Job) {
    // task logic
  }

  @OnQueueEvent('failed')
  onFailed(event: any) {
    console.error(`Task job ${event.jobId} failed: ${event.failedReason}`);
  }
}

Handles the case when jobs are waiting in the queue, useful for empty or slow queues.

NestJS

import { Processor, Process, OnQueueEvent } from '@nestjs/bull';
import { Job } from 'bullmq';

@Processor('emptyQueue')
export class EmptyQueueProcessor {
  @Process()
  async processJob(job: Job) {
    // This queue might be empty sometimes
  }

  @OnQueueEvent('waiting')
  onWaiting(event: any) {
    console.log(`Job ${event.jobId} is waiting in the queue`);
  }
}

Sample Program

This processor handles notification jobs. It logs when a job starts, and listens for completion or failure events to log the results.

NestJS

import { Processor, Process, OnQueueEvent } from '@nestjs/bull';
import { Job } from 'bullmq';

@Processor('notificationQueue')
export class NotificationProcessor {
  @Process()
  async sendNotification(job: Job) {
    console.log(`Processing notification job ${job.id}`);
    // Simulate success or failure
    if (job.data.fail) {
      throw new Error('Notification failed');
    }
    return 'Notification sent';
  }

  @OnQueueEvent('completed')
  onCompleted(event: any) {
    console.log(`Job ${event.jobId} completed successfully.`);
  }

  @OnQueueEvent('failed')
  onFailed(event: any) {
    console.log(`Job ${event.jobId} failed with reason: ${event.failedReason}`);
  }
}

Important Notes

Queue events like 'completed' and 'failed' help you track job status in real time.

Listening to events does not block job processing; it runs alongside your job handler.

Common mistake: forgetting to handle errors inside the job, which can cause silent failures.

Summary

Queue events let you watch what happens to jobs in your queues.

Use @OnQueueEvent decorators to listen for events like 'completed' and 'failed'.

Monitoring queues helps you fix problems and improve your app's background work.

Practice

1. What is the main purpose of using @OnQueueEvent decorators in NestJS queues?

easy

A. To delete jobs from the queue manually

B. To listen and react to specific queue events like job completion or failure

C. To configure the queue's concurrency settings

D. To create new queues dynamically at runtime

Solution

1. Step 1: Understand the role of @OnQueueEvent

   This decorator is used to listen for events emitted by the queue, such as when a job completes or fails.

2. Step 2: Identify what @OnQueueEvent does not do

   It does not create queues, configure concurrency, or delete jobs manually; it only listens to events.

3. Final Answer:

   To listen and react to specific queue events like job completion or failure -> Option B

4. Quick Check:

   @OnQueueEvent listens to events [OK]

Hint: Remember: @OnQueueEvent listens to queue events only [OK]

Common Mistakes:

• Thinking it creates or deletes queues
• Confusing event listening with queue configuration
• Assuming it manages job concurrency

2. Which of the following is the correct syntax to listen for a 'completed' event in a NestJS queue using @OnQueueEvent?

easy

A. @OnQueueEvent('done')
handleDone(job: Job) {
// handle job completion
}

B. @OnQueueEvent('complete')
handleComplete(job: Job) {
// handle job completion
}

C. @OnQueueEvent('completed')
handleCompleted(job: Job) {
// handle job completion
}

D. @OnQueueEvent('finished')
handleFinished(job: Job) {
// handle job completion
}

Solution

1. Step 1: Recall the exact event name for job completion

   The official event name emitted by NestJS queues for job completion is 'completed'.

2. Step 2: Match the syntax with the correct event name

   The syntax must use @OnQueueEvent('completed') to listen properly.

3. Final Answer:

   @OnQueueEvent('completed')\nhandleCompleted(job: Job) {\n // handle job completion\n} -> Option C

4. Quick Check:

   Event name is 'completed' exactly [OK]

Hint: Use exact event names like 'completed' with @OnQueueEvent [OK]

Common Mistakes:

• Using incorrect event names like 'complete' or 'done'
• Misspelling the decorator syntax
• Not passing the event name as a string

3. Given the following code snippet, what will be logged when a job fails in the queue?

@OnQueueEvent('failed')
handleFailed(job: Job, error: Error) {
  console.log(`Job ${job.id} failed with error: ${error.message}`);
}

medium

A. Job <job_id> failed with error: <error_message>

B. Job failed with error: <error_message>

C. Job <job_id> completed successfully

D. Error occurred but no job info logged

Solution

1. Step 1: Analyze the event and handler parameters

   The handler listens to the 'failed' event and receives the job and error objects.

2. Step 2: Understand the console.log output

   The log prints the job's id and the error's message in the format: "Job failed with error: ".

3. Final Answer:

   Job <job_id> failed with error: <error_message> -> Option A

4. Quick Check:

   Failed event logs job id and error message [OK]

Hint: Failed event handlers get job and error objects [OK]

Common Mistakes:

• Ignoring job id in the log message
• Confusing 'failed' with 'completed' event
• Logging incomplete error information

4. You wrote this event listener but it never triggers:

@OnQueueEvent('completed')
handleCompleted() {
  console.log('Job done');
}

What is the likely problem?

medium

A. The event name 'completed' is incorrect

B. You must call the handler manually after job completion

C. The method name must be 'onCompleted' instead of 'handleCompleted'

D. The handler is missing the job parameter to receive event data

Solution

1. Step 1: Check the event handler signature

   Event handlers for 'completed' receive the job object as a parameter, but this method has none.

2. Step 2: Understand why missing parameters cause no trigger

   Without the expected parameter, NestJS cannot properly call the handler, so it never triggers.

3. Final Answer:

   The handler is missing the job parameter to receive event data -> Option D

4. Quick Check:

   Event handlers need correct parameters [OK]

Hint: Always include expected parameters in event handlers [OK]

Common Mistakes:

• Omitting parameters in event handlers
• Changing event names incorrectly
• Thinking handlers must be called manually

5. You want to monitor a NestJS queue to log when jobs are added, completed, or failed. Which combination of event listeners should you implement to cover these events effectively?

hard

A. Use @OnQueueEvent('waiting'), @OnQueueEvent('completed'), and @OnQueueEvent('failed')

B. Use @OnQueueEvent('added'), @OnQueueEvent('done'), and @OnQueueEvent('error')

C. Use @OnQueueEvent('start'), @OnQueueEvent('finish'), and @OnQueueEvent('fail')

D. Use @OnQueueEvent('enqueue'), @OnQueueEvent('success'), and @OnQueueEvent('failure')

Solution

1. Step 1: Identify correct event names for job lifecycle

   In NestJS queues, 'waiting' is emitted when a job is added, 'completed' when done, and 'failed' on failure.

2. Step 2: Verify the other options' event names

   Other options use incorrect or non-existent event names like 'added', 'done', 'error', 'start', 'finish', 'fail', 'enqueue', 'success', 'failure'.

3. Final Answer:

   Use @OnQueueEvent('waiting'), @OnQueueEvent('completed'), and @OnQueueEvent('failed') -> Option A

4. Quick Check:

   Correct event names: waiting, completed, failed [OK]

Hint: Use official event names: waiting, completed, failed [OK]

Common Mistakes:

• Using wrong event names not recognized by NestJS
• Missing the 'waiting' event for job addition
• Confusing 'failed' with 'fail' or 'error'