Piping streams together in Node.js - Mini Project: Build & Apply

Piping Streams Together in Node.js

📖 Scenario: You are building a simple Node.js script that reads data from one file, transforms it, and writes the result to another file using streams.This technique is useful for handling large files efficiently without loading everything into memory.

🎯 Goal: Create a Node.js script that pipes a readable stream from input.txt through a transform stream that converts all text to uppercase, and then pipes it to a writable stream to output.txt.

📋 What You'll Learn

Create a readable stream from input.txt

Create a writable stream to output.txt

Create a transform stream that converts input text to uppercase

Pipe the streams together in the correct order

💡 Why This Matters

🌍 Real World

Piping streams is used in real-world Node.js applications to process large files, such as logs or media files, efficiently without using too much memory.

💼 Career

Understanding streams and piping is essential for backend developers working with Node.js to build scalable and performant data processing applications.

1

Create a readable stream from input.txt

Write a line of code to create a readable stream called readStream from the file input.txt using fs.createReadStream.

Node.js

import fs from 'fs';
// Create a readable stream from 'input.txt'
// Your code here

Need a hint?

Use fs.createReadStream('input.txt') and assign it to readStream.

2

Create a writable stream to output.txt

Add a line of code to create a writable stream called writeStream to the file output.txt using fs.createWriteStream.

Node.js

import fs from 'fs';
const readStream = fs.createReadStream('input.txt');
// Create a writable stream to 'output.txt'
// Your code here

Need a hint?

Use fs.createWriteStream('output.txt') and assign it to writeStream.

3

Create a transform stream to convert text to uppercase

Write code to import Transform from stream and create a transform stream called upperCaseTransform that converts all input chunks to uppercase strings.

Node.js

import fs from 'fs';
const readStream = fs.createReadStream('input.txt');
const writeStream = fs.createWriteStream('output.txt');
// Import Transform and create upperCaseTransform stream
// Your code here

Need a hint?

Create a Transform stream with a transform method that converts chunks to uppercase.

4

Pipe the streams together

Use the pipe method to connect readStream to upperCaseTransform, and then pipe upperCaseTransform to writeStream.

Node.js

import fs from 'fs';
import { Transform } from 'stream';
const readStream = fs.createReadStream('input.txt');
const writeStream = fs.createWriteStream('output.txt');

const upperCaseTransform = new Transform({
  transform(chunk, encoding, callback) {
    this.push(chunk.toString().toUpperCase());
    callback();
  }
});

// Pipe readStream to upperCaseTransform, then to writeStream
// Your code here

Need a hint?

Use readStream.pipe(upperCaseTransform).pipe(writeStream) to connect the streams.