Why streams are needed in Node.js - See It in Action

Why streams are needed in Node.js

📖 Scenario: You are building a Node.js application that processes large files. Loading the entire file into memory at once causes your app to slow down or crash.To solve this, you will learn why streams are needed and how they help handle data efficiently.

🎯 Goal: Build a simple Node.js script that reads a large file using streams instead of loading it all at once.

📋 What You'll Learn

Create a readable stream from a file

Set a chunk size for reading data

Use event listeners to process data chunks

Close the stream properly after reading

💡 Why This Matters

🌍 Real World

Handling large files or data streams efficiently in server applications, such as video streaming or log processing.

💼 Career

Understanding streams is essential for backend developers working with Node.js to build scalable and performant applications.

1

Set up the file path variable

Create a constant called filePath and set it to the string './largefile.txt'.

Node.js

# Your code here

Need a hint?

Use const to declare the file path string.

2

Create a readable stream with chunk size

Import the createReadStream function from the 'fs' module and create a constant called readStream by calling createReadStream with filePath and an options object setting highWaterMark to 16 * 1024 (16 KB chunk size).

Node.js

const filePath = './largefile.txt';
// Your code here

Need a hint?

Use require('fs') to import and set highWaterMark option for chunk size.

3

Add event listeners to process data chunks

Use readStream.on to add a listener for the 'data' event with a callback that takes a parameter chunk. Inside the callback, append the chunk converted to string to a variable called data. Initialize data as an empty string before adding the listener.

Node.js

const filePath = './largefile.txt';
const { createReadStream } = require('fs');
const readStream = createReadStream(filePath, { highWaterMark: 16 * 1024 });

// Your code here

Need a hint?

Initialize data as an empty string and append each chunk converted to string inside the 'data' event callback.

4

Add event listener to handle stream end

Add a listener on readStream for the 'end' event with a callback function that sets a constant message to 'File reading completed.'.

Node.js

const filePath = './largefile.txt';
const { createReadStream } = require('fs');
const readStream = createReadStream(filePath, { highWaterMark: 16 * 1024 });

let data = '';
readStream.on('data', (chunk) => {
  data += chunk.toString();
});

// Your code here

Need a hint?

Use the 'end' event to know when the stream finishes reading.