Load balancing between workers in Node.js - Performance & Optimization

Performance: Load balancing between workers

HIGH IMPACT

This affects how efficiently the server handles incoming requests and distributes CPU work, impacting response time and throughput.

Distributing incoming HTTP requests across multiple Node.js worker processes

Node.js

const cluster = require('cluster');
const http = require('http');
const numCPUs = require('os').cpus().length;

if (cluster.isMaster) {
  cluster.schedulingPolicy = cluster.SCHED_RR;
  for (let i = 0; i < numCPUs; i++) {
    cluster.fork();
  }
  cluster.on('exit', (worker) => {
    cluster.fork(); // replace dead worker
  });
} else {
  http.createServer((req, res) => {
    // handle request
    res.end(`Handled by worker ${process.pid}`);
  }).listen(8000);
}

Using cluster module with multiple workers and round-robin scheduling allows Node.js to distribute load across CPU cores evenly, improving concurrency.

📈 Performance GainReduces request queuing and balances CPU load, lowering INP and improving throughput.

Distributing incoming HTTP requests across multiple Node.js worker processes

Node.js

const cluster = require('cluster');
if (cluster.isMaster) {
  cluster.schedulingPolicy = cluster.SCHED_NONE;
  cluster.fork();
  cluster.fork();
} else {
  require('http').createServer((req, res) => {
    // handle request
    res.end('Handled by worker');
  }).listen(8000);
}

Master process does not balance load; OS assigns requests unevenly causing some workers to be overloaded while others idle.

📉 Performance CostCauses uneven CPU usage and request queuing, increasing average response time and INP.

Performance Comparison

Pattern	CPU Utilization	Request Queuing	Response Time	Verdict
Single worker or unbalanced cluster	Low to uneven	High under load	High latency	[X] Bad
Balanced cluster with multiple workers	High and even	Low	Low latency	[OK] Good

Rendering Pipeline

Load balancing affects the server-side event loop and CPU scheduling, which impacts how quickly requests are processed and responses sent.

→Event Loop

→CPU Scheduling

→Network I/O

⚠️ BottleneckUneven worker load causes some event loops to block longer, delaying response generation.

Core Web Vital Affected

INP

This affects how efficiently the server handles incoming requests and distributes CPU work, impacting response time and throughput.

Optimization Tips

1Use one worker per CPU core for best CPU utilization.

2Restart workers automatically if they crash to maintain availability.

3Avoid single-threaded bottlenecks by distributing requests evenly.

Performance Quiz - 3 Questions

What is the main performance benefit of using multiple Node.js workers with load balancing?

A Improves CPU utilization and reduces request latency

B Reduces memory usage per request

C Eliminates the need for asynchronous code

D Increases the size of the server bundle

DevTools: Performance

How to check: Record a CPU profile while sending concurrent requests; check CPU usage distribution and event loop delays per worker.

What to look for: Balanced CPU usage across workers and minimal event loop blocking indicate good load balancing.

Practice

1. What is the main purpose of using the cluster module in Node.js for load balancing?

easy

A. To spread incoming requests across multiple CPU cores using worker processes

B. To create a single-threaded server that handles all requests

C. To manage database connections efficiently

D. To optimize memory usage by compressing data

Solution

1. Step 1: Understand the role of the cluster module

   The cluster module allows Node.js to create multiple worker processes that share the same server port.

2. Step 2: Identify the purpose of load balancing

   Load balancing means distributing incoming requests evenly across workers to use CPU cores efficiently.

3. Final Answer:

   To spread incoming requests across multiple CPU cores using worker processes -> Option A

4. Quick Check:

   Load balancing = spreading requests across workers [OK]

Hint: Cluster module creates workers to share server load [OK]

Common Mistakes:

• Thinking cluster creates a single-threaded server
• Confusing load balancing with database management
• Assuming cluster compresses data for memory optimization

2. Which of the following is the correct way to check if the current process is the master in a Node.js cluster?

easy

A. if (cluster.isWorker) { /* master code */ }

B. if (cluster.master) { /* master code */ }

C. if (cluster.isMaster) { /* master code */ }

D. if (cluster.worker) { /* master code */ }

Solution

1. Step 1: Recall the cluster API properties

   Node.js cluster module provides isMaster and isWorker boolean properties to identify process roles.

2. Step 2: Identify the correct property for master check

   cluster.isMaster is true if the process is the master, so the condition should use this.

3. Final Answer:

   if (cluster.isMaster) { /* master code */ } -> Option C

4. Quick Check:

   Master check uses cluster.isMaster [OK]

Hint: Use cluster.isMaster to detect master process [OK]

Common Mistakes:

• Using cluster.isWorker to check for master
• Using non-existent properties like cluster.master
• Confusing cluster.worker with cluster.isWorker

3. Consider this Node.js cluster code snippet:

const cluster = require('cluster');
const http = require('http');

if (cluster.isMaster) {
  cluster.fork();
  cluster.fork();
} else {
  http.createServer((req, res) => {
    res.end(`Worker ${process.pid} handled request`);
  }).listen(8000);
}

What will happen when you send multiple requests to port 8000?

medium

A. Requests will be handled by both worker processes, distributing load

B. Only the first worker will handle all requests, the second is idle

C. The master process will handle requests directly

D. The server will crash because multiple workers listen on the same port

Solution

1. Step 1: Understand cluster.fork() behavior

   Calling cluster.fork() creates worker processes that share the same server port.

2. Step 2: Analyze request handling in workers

   Both workers listen on port 8000 and Node.js load balances requests between them automatically.

3. Final Answer:

   Requests will be handled by both worker processes, distributing load -> Option A

4. Quick Check:

   Multiple workers share port and balance requests [OK]

Hint: Multiple forks share port and balance requests [OK]

Common Mistakes:

• Thinking only one worker handles all requests
• Assuming master handles requests directly
• Believing server crashes due to port sharing

4. Given this cluster code snippet, what is the main issue causing the worker to never restart after crashing?

const cluster = require('cluster');

if (cluster.isMaster) {
  cluster.fork();
  cluster.on('exit', (worker, code, signal) => {
    console.log(`Worker ${worker.process.pid} died`);
  });
} else {
  throw new Error('Crash');
}

medium

A. The exit event listener is attached to the wrong object

B. The worker code does not handle errors properly

C. The cluster module is not imported correctly

D. The master does not fork a new worker after exit event

Solution

1. Step 1: Check the exit event handler

   The master listens for 'exit' but only logs the death, it does not fork a new worker.

2. Step 2: Identify missing restart logic

   To restart workers after crash, the master must call cluster.fork() inside the exit event handler.

3. Final Answer:

   The master does not fork a new worker after exit event -> Option D

4. Quick Check:

   Restart requires forking new worker on exit [OK]

Hint: Fork new worker inside exit event to restart [OK]

Common Mistakes:

• Assuming worker error handling restarts process
• Thinking cluster import affects restart
• Believing exit event is attached incorrectly

5. You want to implement a Node.js cluster that balances load across all CPU cores and automatically restarts workers if they crash. Which code snippet correctly achieves this?

hard

A. if (cluster.isMaster) {
cluster.fork();
cluster.on('exit', () => {
console.log('Worker died');
});
} else {
require('http').createServer((req, res) => {
res.end('Hello');
}).listen(3000);
}

B. if (cluster.isMaster) {
const cpuCount = require('os').cpus().length;
for (let i = 0; i < cpuCount; i++) {
cluster.fork();
}
cluster.on('exit', (worker) => {
console.log(`Worker ${worker.process.pid} died, restarting...`);
cluster.fork();
});
} else {
require('http').createServer((req, res) => {
res.end(`Handled by worker ${process.pid}`);
}).listen(3000);
}

C. if (cluster.isWorker) {
const cpuCount = require('os').cpus().length;
for (let i = 0; i < cpuCount; i++) {
cluster.fork();
}
} else {
require('http').createServer((req, res) => {
res.end('Worker running');
}).listen(3000);
}

D. if (cluster.isMaster) {
const cpuCount = require('os').cpus().length;
for (let i = 0; i < cpuCount; i++) {
cluster.fork();
}
} else {
require('http').createServer((req, res) => {
res.end('Worker running');
}).listen(3000);
cluster.on('exit', () => {
cluster.fork();
});
}

Solution

1. Step 1: Fork workers equal to CPU cores in master

   if (cluster.isMaster) { const cpuCount = require('os').cpus().length; for (let i = 0; i < cpuCount; i++) { cluster.fork(); } cluster.on('exit', (worker) => { console.log(`Worker ${worker.process.pid} died, restarting...`); cluster.fork(); }); } else { require('http').createServer((req, res) => { res.end(`Handled by worker ${process.pid}`); }).listen(3000); } correctly uses os.cpus().length to fork that many workers in the master process.

2. Step 2: Restart workers on exit event in master

   if (cluster.isMaster) { const cpuCount = require('os').cpus().length; for (let i = 0; i < cpuCount; i++) { cluster.fork(); } cluster.on('exit', (worker) => { console.log(`Worker ${worker.process.pid} died, restarting...`); cluster.fork(); }); } else { require('http').createServer((req, res) => { res.end(`Handled by worker ${process.pid}`); }).listen(3000); } listens to the 'exit' event on cluster and forks a new worker to replace the dead one.

3. Step 3: Worker creates HTTP server listening on port 3000

   if (cluster.isMaster) { const cpuCount = require('os').cpus().length; for (let i = 0; i < cpuCount; i++) { cluster.fork(); } cluster.on('exit', (worker) => { console.log(`Worker ${worker.process.pid} died, restarting...`); cluster.fork(); }); } else { require('http').createServer((req, res) => { res.end(`Handled by worker ${process.pid}`); }).listen(3000); }'s else block creates the server in workers, which is correct for load balancing.

4. Final Answer:

   Forks workers across all CPU cores and automatically restarts workers if they crash -> Option B

5. Quick Check:

   Fork all CPUs + restart on exit = if (cluster.isMaster) { const cpuCount = require('os').cpus().length; for (let i = 0; i < cpuCount; i++) { cluster.fork(); } cluster.on('exit', (worker) => { console.log(`Worker ${worker.process.pid} died, restarting...`); cluster.fork(); }); } else { require('http').createServer((req, res) => { res.end(`Handled by worker ${process.pid}`); }).listen(3000); } [OK]

Hint: Fork all CPUs in master and restart on exit event [OK]

Common Mistakes:

• Not forking all CPU cores
• Not restarting workers after crash
• Forking workers inside worker process
• Attaching exit listener inside worker instead of master