Handling worker crashes and restart in Node.js - Mini Project: Build & Apply

Handling worker crashes and restart

📖 Scenario: You are building a Node.js application that uses the cluster module to create worker processes. Sometimes, a worker might crash unexpectedly. To keep the app running smoothly, you want to detect when a worker crashes and restart it automatically.

🎯 Goal: Build a Node.js cluster setup that creates one worker process, listens for worker crashes, and restarts the worker automatically when it crashes.

📋 What You'll Learn

Use the Node.js cluster module

Create exactly one worker process

Detect when the worker process exits unexpectedly

Restart the worker automatically after a crash

💡 Why This Matters

🌍 Real World

Many Node.js server applications use clustering to improve performance and reliability. Handling worker crashes and restarting them automatically keeps the app available without manual intervention.

💼 Career

Understanding how to manage worker processes and handle crashes is important for backend developers working with Node.js in production environments.

1

Setup cluster and create one worker

Write code to import the cluster and os modules. Then check if the current process is the master process using cluster.isMaster. If it is, create exactly one worker process using cluster.fork(). If it is a worker process, write a simple HTTP server that listens on port 8000 and responds with 'Hello from worker'.

Node.js

const cluster = require('cluster');
const os = require('os');

// Your code here

Hint

Use cluster.isMaster to check if the current process is the master. Use cluster.fork() to create a worker. In the worker, create a simple HTTP server.

2

Add a crash simulation in the worker

Inside the worker process code, add a timer that crashes the worker after 5 seconds by calling process.exit(1). This simulates a worker crash.

Node.js

const cluster = require('cluster');
const os = require('os');

if (cluster.isMaster) {
    cluster.fork();
} else {
    const http = require('http');
    http.createServer((req, res) => {
        res.writeHead(200);
        res.end('Hello from worker');
    }).listen(8000);

    // Your code here to crash worker after 5 seconds
}

Hint

Use setTimeout to schedule process.exit(1) after 5000 milliseconds.

3

Detect worker exit and restart it

In the master process code, listen for the 'exit' event on cluster. When a worker exits, log a message with console.log that the worker died. Then restart the worker by calling cluster.fork() again.

Node.js

const cluster = require('cluster');
const os = require('os');

if (cluster.isMaster) {
    cluster.fork();

    // Your code here to listen for worker exit and restart
} else {
    const http = require('http');
    http.createServer((req, res) => {
        res.writeHead(200);
        res.end('Hello from worker');
    }).listen(8000);

    setTimeout(() => {
        process.exit(1);
    }, 5000);
}

Hint

Use cluster.on('exit', callback) in the master to detect worker crashes. Inside the callback, log the worker's PID and call cluster.fork() to restart.

4

Add graceful shutdown for workers

In the master process, listen for the 'SIGINT' signal (Ctrl+C). When received, log 'Master shutting down', disconnect the cluster using cluster.disconnect(), and exit the process with process.exit(0). This ensures workers close gracefully.

Node.js

const cluster = require('cluster');
const os = require('os');

if (cluster.isMaster) {
    cluster.fork();

    cluster.on('exit', (worker, code, signal) => {
        console.log(`Worker ${worker.process.pid} died. Restarting...`);
        cluster.fork();
    });

    // Your code here to handle SIGINT and shutdown gracefully
} else {
    const http = require('http');
    http.createServer((req, res) => {
        res.writeHead(200);
        res.end('Hello from worker');
    }).listen(8000);

    setTimeout(() => {
        process.exit(1);
    }, 5000);
}

Hint

Use process.on('SIGINT', callback) in the master to catch Ctrl+C. Inside, log a message, call cluster.disconnect(), and then exit the process.

Practice

1. What is the main purpose of listening to the exit event on a worker in Node.js cluster module?

easy

A. To log the worker's CPU usage

B. To start a new worker automatically

C. To send messages between workers

D. To detect when a worker crashes or stops running

Solution

1. Step 1: Understand the exit event role

   The exit event is triggered when a worker process stops, either normally or due to a crash.

2. Step 2: Identify the purpose of listening to exit

   Listening to exit helps detect unexpected worker crashes so the master can respond.

3. Final Answer:

   To detect when a worker crashes or stops running -> Option D

4. Quick Check:

   exit event = detect crash [OK]

Hint: Remember: exit event means worker stopped or crashed [OK]

Common Mistakes:

• Confusing exit event with message passing
• Thinking exit event starts new workers automatically
• Assuming exit event logs CPU usage

2. Which of the following is the correct way to listen for a worker's exit event in Node.js cluster?

easy

A. cluster.on('exit', worker => { /* handle exit */ });

B. worker.on('exit', () => { /* handle exit */ });

C. worker.listen('exit', () => { /* handle exit */ });

D. process.on('workerExit', () => { /* handle exit */ });

Solution

1. Step 1: Recall event listener syntax on worker

   In Node.js cluster, each worker is an EventEmitter and uses on to listen to events.

2. Step 2: Match correct event and method

   The correct event is exit and the method is on, so worker.on('exit', ...) is correct.

3. Final Answer:

   worker.on('exit', () => { /* handle exit */ }); -> Option B

4. Quick Check:

   Use on with exit on worker [OK]

Hint: Use worker.on('exit', callback) to catch exit events [OK]

Common Mistakes:

• Using .listen instead of .on
• Listening on cluster instead of worker
• Using wrong event name like 'workerExit'

3. Given the code below, what will be logged when a worker crashes?

const cluster = require('cluster');
if (cluster.isMaster) {
  const worker = cluster.fork();
  worker.on('exit', (code, signal) => {
    console.log(`Worker exited with code ${code} and signal ${signal}`);
  });
} else {
  process.exit(1); // Simulate crash
}

medium

A. Worker exited with code null and signal SIGTERM

B. Worker exited with code 0 and signal null

C. Worker exited with code 1 and signal null

D. No output because exit event is not triggered

Solution

1. Step 1: Understand process.exit(1) effect

   Calling process.exit(1) ends the worker with exit code 1, indicating an error.

2. Step 2: Check exit event parameters

   The exit event callback receives the exit code and signal; here signal is null because no signal caused the exit.

3. Final Answer:

   Worker exited with code 1 and signal null -> Option C

4. Quick Check:

   Exit code 1 means crash, signal null if no signal [OK]

Hint: Exit code 1 means crash, signal null if no signal sent [OK]

Common Mistakes:

• Assuming exit code 0 means crash
• Confusing signal with exit code
• Thinking exit event won't fire on crash

4. Identify the error in this code snippet that tries to restart a worker after it crashes:

const cluster = require('cluster');
if (cluster.isMaster) {
  cluster.fork();
  cluster.on('exit', (worker) => {
    console.log('Worker crashed, restarting...');
    cluster.fork();
  });
}

medium

A. The 'exit' event should be listened on 'cluster' but the callback parameter should be (worker, code, signal)

B. The 'exit' event should be listened on 'cluster.workers', not 'cluster'

C. The 'exit' event should be listened on 'cluster' but the callback parameters are wrong

D. The 'exit' event callback parameters are incorrect; it should receive (code, signal)

Solution

1. Step 1: Check where to listen for worker exit

   The 'exit' event is emitted by the cluster module, and the callback receives (worker, code, signal).

2. Step 2: Identify callback parameter mismatch

   The code uses only one parameter (worker), but the event provides three parameters; this can cause confusion or errors.

3. Final Answer:

   The 'exit' event should be listened on 'cluster' but the callback parameter should be (worker, code, signal) -> Option A

4. Quick Check:

   cluster.on('exit', (worker, code, signal)) is correct [OK]

Hint: cluster.on('exit') callback needs (worker, code, signal) parameters [OK]

Common Mistakes:

• Listening on cluster.workers instead of cluster
• Using wrong callback parameters
• Ignoring code and signal parameters

5. You want to ensure your Node.js app automatically restarts a worker if it crashes, but only up to 3 restarts per minute to avoid infinite loops. Which approach best implements this behavior?

hard

A. Use a counter and timestamp in the master process to track restarts; restart only if under limit

B. Restart workers immediately on every exit event without limits

C. Use a setTimeout to delay restarts by 1 minute after each crash

D. Restart workers only if exit code is 0, ignore other exit codes

Solution

1. Step 1: Understand the need to limit restarts

   Unlimited restarts can cause infinite loops if the worker crashes repeatedly.

2. Step 2: Implement a counter and timestamp logic

   Track how many times workers restart within a time window (e.g., 3 restarts per minute) and only restart if under the limit.

3. Final Answer:

   Use a counter and timestamp in the master process to track restarts; restart only if under limit -> Option A

4. Quick Check:

   Limit restarts with counter and time check [OK]

Hint: Count restarts with time checks to avoid infinite loops [OK]

Common Mistakes:

• Restarting without limits causing infinite loops
• Delaying restarts but not counting attempts
• Restarting only on exit code 0 (normal exit)