Buffer and streams relationship in Node.js - Mini Project: Build & Apply

Buffer and Streams Relationship in Node.js

📖 Scenario: You are building a simple Node.js program that reads data from a file using streams and buffers. This is common when handling large files or data chunks in real-world applications like video streaming or file uploads.

🎯 Goal: Learn how to use Buffer and Readable streams together in Node.js to read data in chunks and process it.

📋 What You'll Learn

Create a Buffer to hold data chunks

Set up a Readable stream from a file

Use the data event to receive chunks as buffers

Concatenate buffers to collect full data

End the stream properly

💡 Why This Matters

🌍 Real World

Reading large files or data streams in chunks helps avoid loading everything into memory at once, which is important for performance and scalability.

💼 Career

Understanding buffers and streams is essential for backend developers working with file systems, network data, or real-time data processing in Node.js.

1

Create a Buffer to hold data chunks

Create a variable called dataBuffer and set it to an empty Buffer using Buffer.alloc(0).

Node.js

# Create an empty Buffer called dataBuffer
# Your code here

Need a hint?

Use Buffer.alloc(0) to create an empty buffer.

2

Set up a Readable stream from a file

Import the fs module and create a Readable stream called readStream from the file example.txt using fs.createReadStream.

Node.js

const dataBuffer = Buffer.alloc(0);
// Import fs and create readStream from 'example.txt'
// Your code here

Need a hint?

Use require('fs') and fs.createReadStream('example.txt').

3

Use the data event to receive chunks as buffers

Add a data event listener on readStream with a callback that takes a parameter chunk. Inside the callback, concatenate chunk to dataBuffer using Buffer.concat.

Node.js

const dataBuffer = Buffer.alloc(0);
const fs = require('fs');
const readStream = fs.createReadStream('example.txt');
// Add data event listener on readStream
// Your code here

Need a hint?

Use readStream.on('data', (chunk) => { ... }) and Buffer.concat to add chunks.

4

End the stream properly

Add an end event listener on readStream with a callback that logs 'Stream ended' to the console.

Node.js

let dataBuffer = Buffer.alloc(0);
const fs = require('fs');
const readStream = fs.createReadStream('example.txt');
readStream.on('data', (chunk) => {
    dataBuffer = Buffer.concat([dataBuffer, chunk]);
});
// Add end event listener on readStream
// Your code here

Need a hint?

Use readStream.on('end', () => { console.log('Stream ended') }).