Transform streams for processing in Node.js

Introduction

Transform streams let you change data as it flows through your program. They help process data piece by piece without waiting for everything.

You want to compress or decompress files while reading or writing.

You need to encrypt or decrypt data on the fly.

You want to modify text or data streams, like converting uppercase to lowercase.

You want to filter or transform data from a large file without loading it all at once.

Syntax

Node.js

import { Transform } from 'stream';

const transformStream = new Transform({
  transform(chunk, encoding, callback) {
    // modify chunk here
    const modifiedChunk = chunk; // example placeholder
    this.push(modifiedChunk);
    callback();
  }
});

The transform method runs for each chunk of data.

Call callback() when done processing each chunk.

Examples

This example changes all input text to uppercase.

Node.js

import { Transform } from 'stream';

const upperCaseTransform = new Transform({
  transform(chunk, encoding, callback) {
    this.push(chunk.toString().toUpperCase());
    callback();
  }
});

This example reverses the text in each chunk.

Node.js

import { Transform } from 'stream';

const reverseTransform = new Transform({
  transform(chunk, encoding, callback) {
    const reversed = chunk.toString().split('').reverse().join('');
    this.push(reversed);
    callback();
  }
});

Sample Program

This program reads small pieces of text, changes them to uppercase using a transform stream, and prints the result.

Node.js

import { Transform, Readable, Writable } from 'stream';

// Create a readable stream with some text
const readable = Readable.from(['Hello ', 'world', '!']);

// Transform stream to uppercase
const upperCaseTransform = new Transform({
  transform(chunk, encoding, callback) {
    this.push(chunk.toString().toUpperCase());
    callback();
  }
});

// Writable stream to collect and print output
const writable = new Writable({
  write(chunk, encoding, callback) {
    process.stdout.write(chunk);
    callback();
  }
});

// Pipe streams: readable -> transform -> writable
readable.pipe(upperCaseTransform).pipe(writable);

Important Notes

Transform streams work well with large data because they process chunks, not whole files.

Always call callback() in the transform method to avoid hanging the stream.

You can chain multiple transform streams to do several changes in order.

Summary

Transform streams let you change data as it flows through your program.

Use the transform method to modify each chunk of data.

They help process big data efficiently without loading everything at once.