Transform streams for processing in Node.js - Mini Project: Build & Apply

Transform Streams for Processing Data in Node.js

📖 Scenario: You are building a simple Node.js program that reads text data, processes it by converting all letters to uppercase, and outputs the transformed data. This simulates a real-world task like processing user input or file content before saving or sending it.

🎯 Goal: Create a Node.js transform stream that converts input text to uppercase and outputs the transformed text.

📋 What You'll Learn

Create a readable stream with sample text data

Create a transform stream that converts text to uppercase

Pipe the readable stream through the transform stream

Output the transformed data to the console

💡 Why This Matters

🌍 Real World

Transform streams are used in Node.js to process data on the fly, such as modifying file contents, compressing data, or filtering input before saving or sending it.

💼 Career

Understanding transform streams is important for backend developers working with data processing, file handling, and building efficient network applications.

1

Create a readable stream with sample text

Create a constant called Readable by requiring 'stream' module. Then create a readable stream called inputStream using Readable.from with the array ['hello', ' ', 'world'].

Node.js

const { Readable } = require('stream');
// Create inputStream from array ['hello', ' ', 'world']
// Your code here

Hint

Use Readable.from to create a stream from an array of strings.

2

Create a transform stream to convert text to uppercase

Create a constant called Transform by requiring 'stream' module. Then create a transform stream called upperCaseTransform using new Transform with a transform function that converts the chunk to uppercase string and pushes it.

Node.js

const { Readable, Transform } = require('stream');
const inputStream = Readable.from(['hello', ' ', 'world']);
// Create upperCaseTransform stream that converts chunks to uppercase
// Your code here

Hint

Use the transform method to change each chunk to uppercase and push it forward.

3

Pipe the readable stream through the transform stream

Use inputStream.pipe(upperCaseTransform) and assign the result to a constant called outputStream.

Node.js

const { Readable, Transform } = require('stream');
const inputStream = Readable.from(['hello', ' ', 'world']);

const upperCaseTransform = new Transform({
  transform(chunk, encoding, callback) {
    this.push(chunk.toString().toUpperCase());
    callback();
  }
});

// Pipe inputStream through upperCaseTransform and assign to outputStream
// Your code here

Hint

Use the pipe method to connect streams.

4

Output the transformed data to the console

Add a data event listener on outputStream that receives chunk and writes it to process.stdout.

Node.js

const { Readable, Transform } = require('stream');
const inputStream = Readable.from(['hello', ' ', 'world']);

const upperCaseTransform = new Transform({
  transform(chunk, encoding, callback) {
    this.push(chunk.toString().toUpperCase());
    callback();
  }
});

const outputStream = inputStream.pipe(upperCaseTransform);

// Add data event listener on outputStream to write chunks to process.stdout
// Your code here

Hint

Listen for data events on the output stream and write each chunk to the console.

Practice

1. What is the main purpose of a Transform stream in Node.js?

easy

A. To read data from a file without changing it

B. To write data to a file without reading

C. To modify or transform data chunks as they pass through the stream

D. To buffer all data before processing

Solution

1. Step 1: Understand the role of Transform streams

   Transform streams allow you to change data while it flows, unlike plain readable or writable streams.

2. Step 2: Identify the correct purpose

   Only To modify or transform data chunks as they pass through the stream describes modifying data chunks during streaming, which is the key feature of Transform streams.

3. Final Answer:

   To modify or transform data chunks as they pass through the stream -> Option C

4. Quick Check:

   Transform streams change data on the fly = B [OK]

Hint: Transform streams change data chunks during flow [OK]

Common Mistakes:

• Confusing Transform streams with simple readable or writable streams
• Thinking Transform streams buffer all data before processing
• Assuming Transform streams only read or write without modification

2. Which of the following is the correct way to create a Transform stream using the stream module in Node.js?

easy

A. const { Transform } = require('stream'); const myTransform = new Transform({ transform(chunk, encoding, callback) { callback(null, chunk); } });

B. const { Transform } = require('stream'); const myTransform = Transform();

C. const { Transform } = require('stream'); const myTransform = new Transform();

D. const { Transform } = require('stream'); const myTransform = new Transform({ read() {} });

Solution

1. Step 1: Recall Transform stream creation syntax

   Transform streams require a constructor call with an object containing a transform method to process chunks.

2. Step 2: Check each option

   const { Transform } = require('stream'); const myTransform = new Transform({ transform(chunk, encoding, callback) { callback(null, chunk); } }); correctly uses new Transform with a transform method. const { Transform } = require('stream'); const myTransform = new Transform({ read() {} }); incorrectly uses read instead of transform.

3. Final Answer:

   const { Transform } = require('stream'); const myTransform = new Transform({ transform(chunk, encoding, callback) { callback(null, chunk); } }); -> Option A

4. Quick Check:

   Transform streams need a transform method = D [OK]

Hint: Use new Transform({ transform() }) to create transform streams [OK]

Common Mistakes:

• Forgetting to use the new keyword
• Not providing the transform method
• Using read() instead of transform() in options

3. Given the following Transform stream code, what will be the output when the input chunk is the string "hello"?

const { Transform } = require('stream');
const upperCase = new Transform({
  transform(chunk, encoding, callback) {
    this.push(chunk.toString().toUpperCase());
    callback();
  }
});

upperCase.on('data', data => console.log(data.toString()));
upperCase.write('hello');
upperCase.end();

medium

A. hello

B. HELLO

C. error

D. undefined

Solution

1. Step 1: Analyze the transform function

   The transform method converts the chunk to a string, then to uppercase, and pushes it to the output.

2. Step 2: Determine the output on 'data' event

   The 'data' event logs the transformed chunk, which is "HELLO" in uppercase.

3. Final Answer:

   HELLO -> Option B

4. Quick Check:

   Transform converts input to uppercase = A [OK]

Hint: Transform pushes uppercase chunk, so output is uppercase [OK]

Common Mistakes:

• Expecting original input without transformation
• Confusing push with callback argument
• Missing toString() conversion causing errors

4. Identify the error in the following Transform stream code snippet:

const { Transform } = require('stream');
const reverse = new Transform({
  transform(chunk, encoding, callback) {
    const reversed = chunk.toString().split('').reverse().join('');
    callback(null, reversed);
  }
});

medium

A. The callback should be called with null and the transformed chunk, so no error

B. The transform method is missing the encoding parameter

C. The chunk should not be converted to string before reversing

D. The transformed chunk should be pushed using this.push(), not passed as callback argument

Solution

1. Step 1: Review Transform stream callback usage

   In Transform streams, the transformed data must be pushed using this.push(), not passed as the second argument to callback.

2. Step 2: Identify the mistake in the code

   The code incorrectly calls callback(null, reversed) instead of pushing reversed data and then calling callback().

3. Final Answer:

   The transformed chunk should be pushed using this.push(), not passed as callback argument -> Option D

4. Quick Check:

   Use this.push() for output, callback(null) to signal done = C [OK]

Hint: Push transformed data, then call callback(null) [OK]

Common Mistakes:

• Passing transformed data as callback second argument
• Not calling callback at all
• Ignoring this.push() method

5. You want to create a Transform stream that filters out all chunks containing the word "skip" (case insensitive) and passes through all other chunks unchanged. Which code snippet correctly implements this behavior?

hard

A. const filterSkip = new Transform({ transform(chunk, encoding, callback) { if (chunk.toString().toLowerCase().includes('skip')) { callback(); } else { this.push(chunk); callback(); } } });

B. const filterSkip = new Transform({ transform(chunk, encoding, callback) { if (chunk.includes('skip')) { this.push(chunk); } callback(); } });

C. const filterSkip = new Transform({ transform(chunk, encoding, callback) { if (!chunk.toString().includes('skip')) { this.push(chunk); } callback(null); } });

D. const filterSkip = new Transform({ transform(chunk, encoding, callback) { if (chunk.toString().toLowerCase().indexOf('skip') === -1) { this.push(chunk); callback(); } } });

Solution

1. Step 1: Understand filtering logic

   Chunks containing "skip" (case insensitive) should be ignored (not pushed), others passed through.

2. Step 2: Check each option for correct logic and callback usage

   const filterSkip = new Transform({ transform(chunk, encoding, callback) { if (chunk.toString().toLowerCase().includes('skip')) { callback(); } else { this.push(chunk); callback(); } } }); correctly converts chunk to lowercase, checks for "skip", skips pushing if found, and calls callback in all cases. const filterSkip = new Transform({ transform(chunk, encoding, callback) { if (chunk.includes('skip')) { this.push(chunk); } callback(); } }); pushes chunks containing "skip" (wrong). const filterSkip = new Transform({ transform(chunk, encoding, callback) { if (!chunk.toString().includes('skip')) { this.push(chunk); } callback(null); } }); misses case insensitivity and calls callback with null (acceptable but less consistent). const filterSkip = new Transform({ transform(chunk, encoding, callback) { if (chunk.toString().toLowerCase().indexOf('skip') === -1) { this.push(chunk); callback(); } } }); misses calling callback when chunk contains "skip" (callback not called).

3. Final Answer:

   const filterSkip = new Transform({ transform(chunk, encoding, callback) { if (chunk.toString().toLowerCase().includes('skip')) { callback(); } else { this.push(chunk); callback(); } } }); -> Option A

4. Quick Check:

   Skip chunks with 'skip' word, push others, always call callback = A [OK]

Hint: Call callback always; push only if chunk lacks 'skip' (case insensitive) [OK]

Common Mistakes:

• Not calling callback in all code paths
• Pushing chunks that should be skipped
• Ignoring case sensitivity in filtering