Queue consumers (processors) in NestJS - Performance & Optimization

Performance: Queue consumers (processors)

MEDIUM IMPACT

This affects how fast queued tasks are processed and how much CPU and memory the server uses during processing.

Processing tasks from a queue efficiently

NestJS

import { Processor, Process } from '@nestjs/bull';
import { Job } from 'bull';

@Processor('email')
export class EmailProcessor {
  @Process()
  async handleEmail(job: Job) {
    await this.sendEmailAsync(job.data);
  }

  async sendEmailAsync(data: any) {
    // non-blocking async email sending
    return new Promise(resolve => setTimeout(() => {
      console.log('Email sent', data);
      resolve(true);
    }, 100));
  }
}

Uses asynchronous non-blocking operations allowing other jobs to be processed concurrently, improving throughput and responsiveness.

📈 Performance GainNon-blocking processing reduces CPU spikes and event loop blocking, enabling smoother job handling.

Processing tasks from a queue efficiently

NestJS

import { Processor, Process } from '@nestjs/bull';
import { Job } from 'bull';

@Processor('email')
export class EmailProcessor {
  @Process()
  async handleEmail(job: Job) {
    // heavy synchronous task
    for (let i = 0; i < 1e8; i++) {
      // blocking loop
    }
    console.log('Email sent', job.data);
  }
}

The consumer runs a heavy synchronous task blocking the event loop, causing delays in processing other jobs and increasing response latency.

📉 Performance CostBlocks event loop during processing, causing high CPU usage and delayed job handling.

Performance Comparison

Pattern	CPU Usage	Event Loop Blocking	Memory Usage	Verdict
Synchronous heavy task	High CPU spike	Blocks event loop	Stable	[X] Bad
Asynchronous non-blocking task	Balanced CPU	No blocking	Stable	[OK] Good
Concurrency = 1	Low CPU utilization	No blocking	Stable	[!] OK
Concurrency > 1	Better CPU utilization	No blocking	Stable	[OK] Good
Growing cache without cleanup	Moderate CPU	No blocking	Memory leak	[X] Bad
No cache or cleared cache	Moderate CPU	No blocking	Stable memory	[OK] Good

Rendering Pipeline

Queue consumers process jobs asynchronously, affecting the Node.js event loop and CPU scheduling. Efficient consumers minimize blocking and allow smooth task handling.

→Event Loop

→CPU Scheduling

→Memory Management

⚠️ BottleneckBlocking synchronous code in consumers causes event loop delays and CPU spikes.

Optimization Tips

1Avoid blocking the event loop with synchronous heavy tasks in consumers.

2Use asynchronous code and control concurrency to improve throughput.

3Prevent memory leaks by avoiding unbounded caches or clearing them regularly.

Performance Quiz - 3 Questions

What is the main performance problem with synchronous heavy tasks in queue consumers?

A They block the event loop causing delays in processing other jobs.

B They reduce memory usage drastically.

C They improve concurrency automatically.

D They reduce CPU usage.

DevTools: Performance

How to check: Record a CPU profile while the queue consumer processes jobs. Look for long tasks blocking the event loop or CPU spikes.

What to look for: Long blocking tasks indicate synchronous code; smooth CPU usage and short tasks indicate good async processing.