Reading data with Readable streams in Node.js

Introduction

Readable streams let you get data piece by piece instead of all at once. This helps when working with big files or slow sources.

Reading a large file without loading it all into memory

Receiving data from a network request bit by bit

Processing data as it arrives from a sensor or device

Handling user input streams like keyboard or microphone

Reading data from a database query that returns a stream

Syntax

Node.js

const readable = getReadableStream();

readable.on('data', (chunk) => {
  // handle each chunk of data
});

readable.on('end', () => {
  // all data read
});

The 'data' event gives you chunks of data as they arrive.

The 'end' event tells you when no more data is coming.

Examples

Read a text file chunk by chunk and print each part.

Node.js

const fs = require('node:fs');
const readable = fs.createReadStream('file.txt');

readable.on('data', (chunk) => {
  console.log('Got chunk:', chunk.toString());
});

readable.on('end', () => {
  console.log('Finished reading file');
});

Create a readable stream from an array of strings and print them as they come.

Node.js

const { Readable } = require('node:stream');

const readable = Readable.from(['Hello', ' ', 'world', '!']);

readable.on('data', (chunk) => {
  process.stdout.write(chunk);
});

readable.on('end', () => {
  console.log('\nDone');
});

Sample Program

This program reads the file 'example.txt' in small parts. Each part is printed as it arrives. When done, it prints a message.

Node.js

import { createReadStream } from 'node:fs';

const readable = createReadStream('example.txt', { encoding: 'utf8' });

readable.on('data', (chunk) => {
  console.log(`Received chunk: ${chunk}`);
});

readable.on('end', () => {
  console.log('No more data.');
});

Important Notes

Always handle the 'error' event on streams to catch problems.

Readable streams can be paused and resumed if needed.

Use encoding option to get strings instead of buffers.

Summary

Readable streams let you read data bit by bit, saving memory.

Listen to 'data' for chunks and 'end' when done.

Use streams for large files, network data, or any slow source.

Practice

1. What is the main purpose of a Readable stream in Node.js?

easy

A. To create a new HTTP server

B. To read data piece by piece without loading it all at once

C. To execute JavaScript code asynchronously

D. To write data to a file all at once

Solution

1. Step 1: Understand what a Readable stream does

   A Readable stream reads data in small chunks instead of loading everything at once, which helps save memory.

2. Step 2: Compare options with the purpose

   Only To read data piece by piece without loading it all at once describes reading data piece by piece, which matches the purpose of Readable streams.

3. Final Answer:

   To read data piece by piece without loading it all at once -> Option B

4. Quick Check:

   Readable stream = read data in chunks [OK]

Hint: Readable streams read data bit by bit, not all at once [OK]

Common Mistakes:

• Confusing Readable streams with writing data
• Thinking streams execute code
• Mixing streams with server creation

2. Which of the following is the correct way to listen for data chunks from a Readable stream named stream?

easy

A. stream.on('data', chunk => { console.log(chunk); });

B. stream.emit('data', chunk => { console.log(chunk); });

C. stream.read('data', chunk => { console.log(chunk); });

D. stream.listen('data', chunk => { console.log(chunk); });

Solution

1. Step 1: Recall the event listening method in Node.js streams

   To listen for events, use the on method with the event name and a callback function.

2. Step 2: Match the correct syntax

   stream.on('data', chunk => { console.log(chunk); }); uses stream.on('data', callback), which is the correct way to listen for data chunks.

3. Final Answer:

   stream.on('data', chunk => { console.log(chunk); }); -> Option A

4. Quick Check:

   Use .on() to listen to stream events [OK]

Hint: Use .on('data', callback) to read chunks from streams [OK]

Common Mistakes:

• Using emit instead of on to listen
• Using non-existent methods like read or listen
• Confusing event listening with emitting

3. What will the following code output if the file example.txt contains the text "Hello World"?

const fs = require('fs');
const stream = fs.createReadStream('example.txt');
stream.on('data', chunk => {
  console.log(chunk.toString());
});
stream.on('end', () => {
  console.log('Done reading');
});

medium

A. Hello World
Done reading

B. Buffer data
Done reading

C. Hello WorldDone reading

D. SyntaxError

Solution

1. Step 1: Understand the stream reading process

   The stream reads the file in chunks and emits 'data' events. Each chunk is a Buffer, converted to string with toString().

2. Step 2: Analyze the output

   The console logs the text "Hello World" followed by a newline, then logs "Done reading" when the stream ends.

3. Final Answer:

   Hello World Done reading -> Option A

4. Quick Check:

   Stream reads text chunks and logs 'Done reading' at end [OK]

Hint: Streams emit 'data' chunks as Buffers; convert to string to read [OK]

Common Mistakes:

• Expecting raw Buffer output without conversion
• Missing the 'end' event message
• Confusing output formatting

4. Identify the error in this code snippet that reads data from a stream:

const fs = require('fs');
const stream = fs.createReadStream('file.txt');
stream.on('data', (chunk) => {
  console.log(chunk);
});
stream.on('finish', () => {
  console.log('Stream finished');
});

medium

A. The 'data' event callback is missing a parameter

B. The stream should be paused before reading

C. The 'finish' event does not exist on Readable streams

D. The file path must be absolute

Solution

1. Step 1: Check event names for Readable streams

   Readable streams emit 'end' when done, not 'finish'. 'finish' is for Writable streams.

2. Step 2: Identify the incorrect event usage

   The code listens for 'finish', which will never fire on a Readable stream, causing the message to never appear.

3. Final Answer:

   The 'finish' event does not exist on Readable streams -> Option C

4. Quick Check:

   Readable streams use 'end', not 'finish' event [OK]

Hint: 'finish' is for writing; use 'end' for reading streams [OK]

Common Mistakes:

• Using 'finish' event on Readable streams
• Forgetting to handle 'end' event
• Assuming 'data' event has no parameters

5. You want to read a large file using a Readable stream and count how many times the word "node" appears, case-insensitive. Which approach correctly handles data chunks to count occurrences without missing matches across chunk boundaries?

hard

A. Convert each chunk to string and count occurrences immediately, ignoring chunk edges

B. Count occurrences in each chunk separately without storing leftover text

C. Use a writable stream instead of a readable stream for counting

D. Concatenate chunks into a string, then count occurrences after 'end' event

Solution

1. Step 1: Understand chunk boundaries in streams

   Chunks may split words, so counting in each chunk separately can miss matches that cross chunk edges.

2. Step 2: Choose a method to avoid missing matches

   Concatenating all chunks into one string and counting after the 'end' event ensures all occurrences are counted correctly.

3. Final Answer:

   Concatenate chunks into a string, then count occurrences after 'end' event -> Option D

4. Quick Check:

   Count after full data read to avoid split word misses [OK]

Hint: Join chunks first, then count words to avoid split matches [OK]

Common Mistakes:

• Counting words in each chunk ignoring splits
• Using writable streams for reading tasks
• Not handling case-insensitive matching