Transform streams for processing in Node.js

Introduction

Transform streams let you change data as it flows through your program. They help process data piece by piece without waiting for everything.

You want to compress or decompress files while reading or writing.

You need to encrypt or decrypt data on the fly.

You want to modify text or data streams, like converting uppercase to lowercase.

You want to filter or transform data from a large file without loading it all at once.

Syntax

Node.js

import { Transform } from 'stream';

const transformStream = new Transform({
  transform(chunk, encoding, callback) {
    // modify chunk here
    const modifiedChunk = chunk; // example placeholder
    this.push(modifiedChunk);
    callback();
  }
});

The transform method runs for each chunk of data.

Call callback() when done processing each chunk.

Examples

This example changes all input text to uppercase.

Node.js

import { Transform } from 'stream';

const upperCaseTransform = new Transform({
  transform(chunk, encoding, callback) {
    this.push(chunk.toString().toUpperCase());
    callback();
  }
});

This example reverses the text in each chunk.

Node.js

import { Transform } from 'stream';

const reverseTransform = new Transform({
  transform(chunk, encoding, callback) {
    const reversed = chunk.toString().split('').reverse().join('');
    this.push(reversed);
    callback();
  }
});

Sample Program

This program reads small pieces of text, changes them to uppercase using a transform stream, and prints the result.

Node.js

import { Transform, Readable, Writable } from 'stream';

// Create a readable stream with some text
const readable = Readable.from(['Hello ', 'world', '!']);

// Transform stream to uppercase
const upperCaseTransform = new Transform({
  transform(chunk, encoding, callback) {
    this.push(chunk.toString().toUpperCase());
    callback();
  }
});

// Writable stream to collect and print output
const writable = new Writable({
  write(chunk, encoding, callback) {
    process.stdout.write(chunk);
    callback();
  }
});

// Pipe streams: readable -> transform -> writable
readable.pipe(upperCaseTransform).pipe(writable);

Important Notes

Transform streams work well with large data because they process chunks, not whole files.

Always call callback() in the transform method to avoid hanging the stream.

You can chain multiple transform streams to do several changes in order.

Summary

Transform streams let you change data as it flows through your program.

Use the transform method to modify each chunk of data.

They help process big data efficiently without loading everything at once.

Practice

1. What is the main purpose of a Transform stream in Node.js?

easy

A. To read data from a file without changing it

B. To write data to a file without reading

C. To modify or transform data chunks as they pass through the stream

D. To buffer all data before processing

Solution

1. Step 1: Understand the role of Transform streams

   Transform streams allow you to change data while it flows, unlike plain readable or writable streams.

2. Step 2: Identify the correct purpose

   Only To modify or transform data chunks as they pass through the stream describes modifying data chunks during streaming, which is the key feature of Transform streams.

3. Final Answer:

   To modify or transform data chunks as they pass through the stream -> Option C

4. Quick Check:

   Transform streams change data on the fly = B [OK]

Hint: Transform streams change data chunks during flow [OK]

Common Mistakes:

• Confusing Transform streams with simple readable or writable streams
• Thinking Transform streams buffer all data before processing
• Assuming Transform streams only read or write without modification

2. Which of the following is the correct way to create a Transform stream using the stream module in Node.js?

easy

A. const { Transform } = require('stream'); const myTransform = new Transform({ transform(chunk, encoding, callback) { callback(null, chunk); } });

B. const { Transform } = require('stream'); const myTransform = Transform();

C. const { Transform } = require('stream'); const myTransform = new Transform();

D. const { Transform } = require('stream'); const myTransform = new Transform({ read() {} });

Solution

1. Step 1: Recall Transform stream creation syntax

   Transform streams require a constructor call with an object containing a transform method to process chunks.

2. Step 2: Check each option

   const { Transform } = require('stream'); const myTransform = new Transform({ transform(chunk, encoding, callback) { callback(null, chunk); } }); correctly uses new Transform with a transform method. const { Transform } = require('stream'); const myTransform = new Transform({ read() {} }); incorrectly uses read instead of transform.

3. Final Answer:

   const { Transform } = require('stream'); const myTransform = new Transform({ transform(chunk, encoding, callback) { callback(null, chunk); } }); -> Option A

4. Quick Check:

   Transform streams need a transform method = D [OK]

Hint: Use new Transform({ transform() }) to create transform streams [OK]

Common Mistakes:

• Forgetting to use the new keyword
• Not providing the transform method
• Using read() instead of transform() in options

3. Given the following Transform stream code, what will be the output when the input chunk is the string "hello"?

const { Transform } = require('stream');
const upperCase = new Transform({
  transform(chunk, encoding, callback) {
    this.push(chunk.toString().toUpperCase());
    callback();
  }
});

upperCase.on('data', data => console.log(data.toString()));
upperCase.write('hello');
upperCase.end();

medium

A. hello

B. HELLO

C. error

D. undefined

Solution

1. Step 1: Analyze the transform function

   The transform method converts the chunk to a string, then to uppercase, and pushes it to the output.

2. Step 2: Determine the output on 'data' event

   The 'data' event logs the transformed chunk, which is "HELLO" in uppercase.

3. Final Answer:

   HELLO -> Option B

4. Quick Check:

   Transform converts input to uppercase = A [OK]

Hint: Transform pushes uppercase chunk, so output is uppercase [OK]

Common Mistakes:

• Expecting original input without transformation
• Confusing push with callback argument
• Missing toString() conversion causing errors

4. Identify the error in the following Transform stream code snippet:

const { Transform } = require('stream');
const reverse = new Transform({
  transform(chunk, encoding, callback) {
    const reversed = chunk.toString().split('').reverse().join('');
    callback(null, reversed);
  }
});

medium

A. The callback should be called with null and the transformed chunk, so no error

B. The transform method is missing the encoding parameter

C. The chunk should not be converted to string before reversing

D. The transformed chunk should be pushed using this.push(), not passed as callback argument

Solution

1. Step 1: Review Transform stream callback usage

   In Transform streams, the transformed data must be pushed using this.push(), not passed as the second argument to callback.

2. Step 2: Identify the mistake in the code

   The code incorrectly calls callback(null, reversed) instead of pushing reversed data and then calling callback().

3. Final Answer:

   The transformed chunk should be pushed using this.push(), not passed as callback argument -> Option D

4. Quick Check:

   Use this.push() for output, callback(null) to signal done = C [OK]

Hint: Push transformed data, then call callback(null) [OK]

Common Mistakes:

• Passing transformed data as callback second argument
• Not calling callback at all
• Ignoring this.push() method

5. You want to create a Transform stream that filters out all chunks containing the word "skip" (case insensitive) and passes through all other chunks unchanged. Which code snippet correctly implements this behavior?

hard

A. const filterSkip = new Transform({ transform(chunk, encoding, callback) { if (chunk.toString().toLowerCase().includes('skip')) { callback(); } else { this.push(chunk); callback(); } } });

B. const filterSkip = new Transform({ transform(chunk, encoding, callback) { if (chunk.includes('skip')) { this.push(chunk); } callback(); } });

C. const filterSkip = new Transform({ transform(chunk, encoding, callback) { if (!chunk.toString().includes('skip')) { this.push(chunk); } callback(null); } });

D. const filterSkip = new Transform({ transform(chunk, encoding, callback) { if (chunk.toString().toLowerCase().indexOf('skip') === -1) { this.push(chunk); callback(); } } });

Solution

1. Step 1: Understand filtering logic

   Chunks containing "skip" (case insensitive) should be ignored (not pushed), others passed through.

2. Step 2: Check each option for correct logic and callback usage

   const filterSkip = new Transform({ transform(chunk, encoding, callback) { if (chunk.toString().toLowerCase().includes('skip')) { callback(); } else { this.push(chunk); callback(); } } }); correctly converts chunk to lowercase, checks for "skip", skips pushing if found, and calls callback in all cases. const filterSkip = new Transform({ transform(chunk, encoding, callback) { if (chunk.includes('skip')) { this.push(chunk); } callback(); } }); pushes chunks containing "skip" (wrong). const filterSkip = new Transform({ transform(chunk, encoding, callback) { if (!chunk.toString().includes('skip')) { this.push(chunk); } callback(null); } }); misses case insensitivity and calls callback with null (acceptable but less consistent). const filterSkip = new Transform({ transform(chunk, encoding, callback) { if (chunk.toString().toLowerCase().indexOf('skip') === -1) { this.push(chunk); callback(); } } }); misses calling callback when chunk contains "skip" (callback not called).

3. Final Answer:

   const filterSkip = new Transform({ transform(chunk, encoding, callback) { if (chunk.toString().toLowerCase().includes('skip')) { callback(); } else { this.push(chunk); callback(); } } }); -> Option A

4. Quick Check:

   Skip chunks with 'skip' word, push others, always call callback = A [OK]

Hint: Call callback always; push only if chunk lacks 'skip' (case insensitive) [OK]

Common Mistakes:

• Not calling callback in all code paths
• Pushing chunks that should be skipped
• Ignoring case sensitivity in filtering