Stream types (Readable, Writable, Transform, Duplex) in Node.js

Introduction

Streams help handle data piece by piece instead of all at once. This saves memory and lets programs work faster with big data.

Reading a large file without loading it all into memory

Sending data over the internet bit by bit

Transforming data on the fly, like compressing or encrypting

Building a chat app where data flows both ways

Syntax

Node.js

const { Readable, Writable, Transform, Duplex } = require('stream');

These are classes you can use to create different stream types.

Each stream type has specific methods to implement for your data flow.

Examples

A Readable stream that sends 'Hello' then ends.

Node.js

const readable = new Readable({
  read(size) {
    this.push('Hello');
    this.push(null); // No more data
  }
});

A Writable stream that prints whatever it receives.

Node.js

const writable = new Writable({
  write(chunk, encoding, callback) {
    console.log(chunk.toString());
    callback();
  }
});

A Transform stream that changes input text to uppercase.

Node.js

const transform = new Transform({
  transform(chunk, encoding, callback) {
    this.push(chunk.toString().toUpperCase());
    callback();
  }
});

A Duplex stream that can read and write data.

Node.js

const duplex = new Duplex({
  read(size) {
    this.push('Data from read side');
    this.push(null);
  },
  write(chunk, encoding, callback) {
    console.log('Received:', chunk.toString());
    callback();
  }
});

Sample Program

This program creates three streams. The readable stream sends numbers 1, 2, and 3. The transform stream doubles each number. The writable stream prints the doubled numbers. The streams are connected so data flows from readable to transform to writable.

Node.js

const { Readable, Writable, Transform } = require('stream');

// Readable stream sends numbers 1 to 3
const readable = new Readable({
  read() {
    if (!this.current) this.current = 1;
    if (this.current <= 3) {
      this.push(this.current.toString());
      this.current++;
    } else {
      this.push(null); // End
    }
  }
});

// Transform stream doubles the number
const transform = new Transform({
  transform(chunk, encoding, callback) {
    const num = Number(chunk.toString());
    this.push((num * 2).toString());
    callback();
  }
});

// Writable stream prints the result
const writable = new Writable({
  write(chunk, encoding, callback) {
    console.log('Output:', chunk.toString());
    callback();
  }
});

// Connect streams: readable -> transform -> writable
readable.pipe(transform).pipe(writable);

Important Notes

Always call callback() in writable and transform streams to signal you finished processing.

Use this.push(null) in readable streams to mark the end of data.

Duplex streams combine readable and writable in one object for two-way data flow.

Summary

Readable streams provide data piece by piece.

Writable streams accept and process data.

Transform streams change data while passing it along.

Duplex streams can both read and write data.

Practice

1. Which type of Node.js stream is designed to only provide data piece by piece for reading?

easy

A. Transform stream

B. Readable stream

C. Writable stream

D. Duplex stream

Solution

1. Step 1: Understand stream roles

   Readable streams are designed to emit data chunks for consumption.

2. Step 2: Match stream type to description

   Only readable streams provide data piece by piece without accepting input.

3. Final Answer:

   Readable stream -> Option B

4. Quick Check:

   Readable = provides data [OK]

Hint: Readable streams only output data, no input accepted [OK]

Common Mistakes:

• Confusing writable streams as data providers
• Thinking transform streams only read data
• Mixing duplex with readable-only behavior

2. Which of the following is the correct way to create a writable stream in Node.js?

easy

A. const stream = new Readable();

B. const stream = new Duplex();

C. const stream = new Transform();

D. const stream = new Writable({ write(chunk, encoding, callback) { callback(); } });

Solution

1. Step 1: Identify writable stream creation

   Writable streams require a write method to handle incoming data chunks.

2. Step 2: Check options for correct syntax

   const stream = new Writable({ write(chunk, encoding, callback) { callback(); } }); correctly creates a Writable stream with a write method and callback.

3. Final Answer:

   const stream = new Writable({ write(chunk, encoding, callback) { callback(); } }); -> Option D

4. Quick Check:

   Writable needs write() method [OK]

Hint: Writable streams need a write() method in options [OK]

Common Mistakes:

• Using Readable constructor for writable stream
• Omitting write method in Writable options
• Confusing Transform or Duplex constructors

3. What will be the output of the following code snippet?

const { Transform } = require('stream');
const upperCase = new Transform({
  transform(chunk, encoding, callback) {
    this.push(chunk.toString().toUpperCase());
    callback();
  }
});

upperCase.on('data', data => console.log(data.toString()));
upperCase.write('hello');
upperCase.end();

medium

A. HELLO

B. hello

C. undefined

D. Error: callback not called

Solution

1. Step 1: Understand Transform stream behavior

   The transform method converts input chunk to uppercase and pushes it forward.

2. Step 2: Analyze event and output

   The 'data' event logs the transformed chunk, which is uppercase 'HELLO'.

3. Final Answer:

   HELLO -> Option A

4. Quick Check:

   Transform changes data to uppercase [OK]

Hint: Transform streams modify data before output [OK]

Common Mistakes:

• Expecting original lowercase output
• Forgetting to call callback() in transform
• Assuming no output without explicit read

4. Identify the error in this Duplex stream implementation:

const { Duplex } = require('stream');
const duplex = new Duplex({
  read(size) {
    this.push('data');
  },
  write(chunk, encoding, callback) {
    console.log(chunk.toString());
    callback();
  }
});
duplex.on('data', data => console.log('Received:', data.toString()));
duplex.write('hello');

medium

A. The Duplex constructor is missing the object mode option

B. The write method is missing the callback call

C. The read method should push null to signal end of data

D. The duplex stream cannot both read and write

Solution

1. Step 1: Check read method behavior

   The read method pushes 'data' but never signals end by pushing null.

2. Step 2: Understand stream end signaling

   Streams must push null to indicate no more data; missing here causes hanging.

3. Final Answer:

   The read method should push null to signal end of data -> Option C

4. Quick Check:

   read() must push null to end stream [OK]

Hint: read() must push null to end data stream [OK]

Common Mistakes:

• Not pushing null in read method
• Forgetting callback in write method
• Assuming Duplex can't read and write

5. You want to create a custom Duplex stream that reads numbers from 1 to 3 and writes their squares. Which approach correctly implements this behavior?

hard

A. Push numbers 1 to 3 in read(), write() logs squares, and push null after 3

B. Push squares in read(), write() pushes numbers 1 to 3, no null needed

C. Use Transform stream instead of Duplex for this task

D. Only Writable stream can handle both reading and writing

Solution

1. Step 1: Understand Duplex stream roles

   Duplex streams can read and write independently; read() pushes data, write() processes input.

2. Step 2: Match behavior to implementation

   Push numbers 1 to 3 in read(), write() logs squares, and push null after last number to end stream.

3. Final Answer:

   Push numbers 1 to 3 in read(), write() logs squares, and push null after 3 -> Option A

4. Quick Check:

   Duplex reads numbers, writes squares [OK]

Hint: Duplex streams read and write independently [OK]

Common Mistakes:

• Confusing Transform with Duplex for this task
• Not pushing null to end read stream
• Thinking Writable streams can read data