Reading data with Readable streams in Node.js - Mini Project: Build & Apply

Reading data with Readable streams

📖 Scenario: You are building a simple Node.js program that reads data from a file using a Readable stream. This is useful when working with large files or data sources that send data in chunks.

🎯 Goal: Learn how to create a Readable stream from a file, set up a data event listener to read chunks, and handle the end of the stream.

📋 What You'll Learn

Create a Readable stream from a file named example.txt

Set a variable to count the total bytes read

Use the data event to add chunk sizes to the total bytes

Use the end event to mark the completion of reading

💡 Why This Matters

🌍 Real World

Reading large files or data streams in chunks helps avoid loading everything into memory at once, which is important for performance and scalability.

💼 Career

Understanding streams is essential for backend developers working with file systems, network data, or any large data processing in Node.js.

1

Create a Readable stream from a file

Write code to import the fs module and create a Readable stream called readStream from the file example.txt using fs.createReadStream.

Node.js

const fs = require('fs');
// Your code here

Hint

Use require('fs') to import the file system module. Then use fs.createReadStream('example.txt') to create the stream.

2

Set up a variable to count bytes

Add a variable called totalBytes and set it to 0. This will keep track of how many bytes have been read from the stream.

Node.js

const fs = require('fs');
const readStream = fs.createReadStream('example.txt');
// Your code here

Hint

Use let totalBytes = 0; to create the counter variable.

3

Add a data event listener to count bytes

Use readStream.on('data', chunk => { ... }) to listen for data chunks. Inside the callback, add chunk.length to totalBytes.

Node.js

const fs = require('fs');
const readStream = fs.createReadStream('example.txt');
let totalBytes = 0;
// Your code here

Hint

Use the data event to get chunks and add their length to totalBytes.

4

Add an end event listener to finish reading

Add a listener for the end event on readStream using readStream.on('end', () => { ... }). Inside, add a comment // Reading finished to mark the end of the stream.

Node.js

const fs = require('fs');
const readStream = fs.createReadStream('example.txt');
let totalBytes = 0;
readStream.on('data', chunk => {
    totalBytes += chunk.length;
});
// Your code here

Hint

Use the end event to know when the stream has finished sending data.

Practice

1. What is the main purpose of a Readable stream in Node.js?

easy

A. To create a new HTTP server

B. To read data piece by piece without loading it all at once

C. To execute JavaScript code asynchronously

D. To write data to a file all at once

Solution

1. Step 1: Understand what a Readable stream does

   A Readable stream reads data in small chunks instead of loading everything at once, which helps save memory.

2. Step 2: Compare options with the purpose

   Only To read data piece by piece without loading it all at once describes reading data piece by piece, which matches the purpose of Readable streams.

3. Final Answer:

   To read data piece by piece without loading it all at once -> Option B

4. Quick Check:

   Readable stream = read data in chunks [OK]

Hint: Readable streams read data bit by bit, not all at once [OK]

Common Mistakes:

• Confusing Readable streams with writing data
• Thinking streams execute code
• Mixing streams with server creation

2. Which of the following is the correct way to listen for data chunks from a Readable stream named stream?

easy

A. stream.on('data', chunk => { console.log(chunk); });

B. stream.emit('data', chunk => { console.log(chunk); });

C. stream.read('data', chunk => { console.log(chunk); });

D. stream.listen('data', chunk => { console.log(chunk); });

Solution

1. Step 1: Recall the event listening method in Node.js streams

   To listen for events, use the on method with the event name and a callback function.

2. Step 2: Match the correct syntax

   stream.on('data', chunk => { console.log(chunk); }); uses stream.on('data', callback), which is the correct way to listen for data chunks.

3. Final Answer:

   stream.on('data', chunk => { console.log(chunk); }); -> Option A

4. Quick Check:

   Use .on() to listen to stream events [OK]

Hint: Use .on('data', callback) to read chunks from streams [OK]

Common Mistakes:

• Using emit instead of on to listen
• Using non-existent methods like read or listen
• Confusing event listening with emitting

3. What will the following code output if the file example.txt contains the text "Hello World"?

const fs = require('fs');
const stream = fs.createReadStream('example.txt');
stream.on('data', chunk => {
  console.log(chunk.toString());
});
stream.on('end', () => {
  console.log('Done reading');
});

medium

A. Hello World
Done reading

B. Buffer data
Done reading

C. Hello WorldDone reading

D. SyntaxError

Solution

1. Step 1: Understand the stream reading process

   The stream reads the file in chunks and emits 'data' events. Each chunk is a Buffer, converted to string with toString().

2. Step 2: Analyze the output

   The console logs the text "Hello World" followed by a newline, then logs "Done reading" when the stream ends.

3. Final Answer:

   Hello World Done reading -> Option A

4. Quick Check:

   Stream reads text chunks and logs 'Done reading' at end [OK]

Hint: Streams emit 'data' chunks as Buffers; convert to string to read [OK]

Common Mistakes:

• Expecting raw Buffer output without conversion
• Missing the 'end' event message
• Confusing output formatting

4. Identify the error in this code snippet that reads data from a stream:

const fs = require('fs');
const stream = fs.createReadStream('file.txt');
stream.on('data', (chunk) => {
  console.log(chunk);
});
stream.on('finish', () => {
  console.log('Stream finished');
});

medium

A. The 'data' event callback is missing a parameter

B. The stream should be paused before reading

C. The 'finish' event does not exist on Readable streams

D. The file path must be absolute

Solution

1. Step 1: Check event names for Readable streams

   Readable streams emit 'end' when done, not 'finish'. 'finish' is for Writable streams.

2. Step 2: Identify the incorrect event usage

   The code listens for 'finish', which will never fire on a Readable stream, causing the message to never appear.

3. Final Answer:

   The 'finish' event does not exist on Readable streams -> Option C

4. Quick Check:

   Readable streams use 'end', not 'finish' event [OK]

Hint: 'finish' is for writing; use 'end' for reading streams [OK]

Common Mistakes:

• Using 'finish' event on Readable streams
• Forgetting to handle 'end' event
• Assuming 'data' event has no parameters

5. You want to read a large file using a Readable stream and count how many times the word "node" appears, case-insensitive. Which approach correctly handles data chunks to count occurrences without missing matches across chunk boundaries?

hard

A. Convert each chunk to string and count occurrences immediately, ignoring chunk edges

B. Count occurrences in each chunk separately without storing leftover text

C. Use a writable stream instead of a readable stream for counting

D. Concatenate chunks into a string, then count occurrences after 'end' event

Solution

1. Step 1: Understand chunk boundaries in streams

   Chunks may split words, so counting in each chunk separately can miss matches that cross chunk edges.

2. Step 2: Choose a method to avoid missing matches

   Concatenating all chunks into one string and counting after the 'end' event ensures all occurrences are counted correctly.

3. Final Answer:

   Concatenate chunks into a string, then count occurrences after 'end' event -> Option D

4. Quick Check:

   Count after full data read to avoid split word misses [OK]

Hint: Join chunks first, then count words to avoid split matches [OK]

Common Mistakes:

• Counting words in each chunk ignoring splits
• Using writable streams for reading tasks
• Not handling case-insensitive matching