Reading data with Readable streams in Node.js - Performance & Optimization

Performance: Reading data with Readable streams

MEDIUM IMPACT

This concept affects how fast data is processed and delivered to the application, impacting input responsiveness and memory usage.

Reading large files or data chunks efficiently

Node.js

const fs = require('fs');
const stream = fs.createReadStream('largefile.txt', { encoding: 'utf8' });
stream.on('data', chunk => console.log(chunk));

Reads data in small chunks asynchronously, keeping memory low and event loop free for other tasks.

📈 Performance GainNon-blocking, low memory usage, smooth input responsiveness

Reading large files or data chunks efficiently

Node.js

const fs = require('fs');
const data = fs.readFileSync('largefile.txt', 'utf8');
console.log(data);

Reads entire file into memory blocking the event loop until done, causing slow response and high memory use.

📉 Performance CostBlocks event loop during read, high memory usage proportional to file size

Performance Comparison

Pattern	DOM Operations	Reflows	Paint Cost	Verdict
Synchronous readFileSync	N/A	N/A	Blocks event loop, delays UI updates	[X] Bad
Asynchronous Readable stream	N/A	N/A	Non-blocking, smooth UI responsiveness	[OK] Good

Rendering Pipeline

Readable streams process data in chunks, allowing the event loop to handle other tasks between chunks, improving responsiveness.

→Data Fetching

→Event Loop

→Memory Management

⚠️ BottleneckBlocking synchronous reads that freeze the event loop

Core Web Vital Affected

INP

This concept affects how fast data is processed and delivered to the application, impacting input responsiveness and memory usage.

Optimization Tips

1Avoid synchronous file reads for large data to prevent blocking the event loop.

2Use Readable streams to process data in manageable chunks asynchronously.

3Monitor event loop responsiveness to ensure smooth app performance.

Performance Quiz - 3 Questions

What is the main performance benefit of using Readable streams over synchronous file reads in Node.js?

A They reduce the file size on disk

B They load the entire file into memory faster

C They read data in small chunks without blocking the event loop

D They automatically cache data in the browser

DevTools: Performance

How to check: Record a performance profile while running your Node.js app reading data; look for long blocking tasks and event loop delays.

What to look for: Long blocking periods indicate synchronous reads; short, spread out tasks indicate efficient streaming.

Practice

1. What is the main purpose of a Readable stream in Node.js?

easy

A. To create a new HTTP server

B. To read data piece by piece without loading it all at once

C. To execute JavaScript code asynchronously

D. To write data to a file all at once

Solution

1. Step 1: Understand what a Readable stream does

   A Readable stream reads data in small chunks instead of loading everything at once, which helps save memory.

2. Step 2: Compare options with the purpose

   Only To read data piece by piece without loading it all at once describes reading data piece by piece, which matches the purpose of Readable streams.

3. Final Answer:

   To read data piece by piece without loading it all at once -> Option B

4. Quick Check:

   Readable stream = read data in chunks [OK]

Hint: Readable streams read data bit by bit, not all at once [OK]

Common Mistakes:

• Confusing Readable streams with writing data
• Thinking streams execute code
• Mixing streams with server creation

2. Which of the following is the correct way to listen for data chunks from a Readable stream named stream?

easy

A. stream.on('data', chunk => { console.log(chunk); });

B. stream.emit('data', chunk => { console.log(chunk); });

C. stream.read('data', chunk => { console.log(chunk); });

D. stream.listen('data', chunk => { console.log(chunk); });

Solution

1. Step 1: Recall the event listening method in Node.js streams

   To listen for events, use the on method with the event name and a callback function.

2. Step 2: Match the correct syntax

   stream.on('data', chunk => { console.log(chunk); }); uses stream.on('data', callback), which is the correct way to listen for data chunks.

3. Final Answer:

   stream.on('data', chunk => { console.log(chunk); }); -> Option A

4. Quick Check:

   Use .on() to listen to stream events [OK]

Hint: Use .on('data', callback) to read chunks from streams [OK]

Common Mistakes:

• Using emit instead of on to listen
• Using non-existent methods like read or listen
• Confusing event listening with emitting

3. What will the following code output if the file example.txt contains the text "Hello World"?

const fs = require('fs');
const stream = fs.createReadStream('example.txt');
stream.on('data', chunk => {
  console.log(chunk.toString());
});
stream.on('end', () => {
  console.log('Done reading');
});

medium

A. Hello World
Done reading

B. Buffer data
Done reading

C. Hello WorldDone reading

D. SyntaxError

Solution

1. Step 1: Understand the stream reading process

   The stream reads the file in chunks and emits 'data' events. Each chunk is a Buffer, converted to string with toString().

2. Step 2: Analyze the output

   The console logs the text "Hello World" followed by a newline, then logs "Done reading" when the stream ends.

3. Final Answer:

   Hello World Done reading -> Option A

4. Quick Check:

   Stream reads text chunks and logs 'Done reading' at end [OK]

Hint: Streams emit 'data' chunks as Buffers; convert to string to read [OK]

Common Mistakes:

• Expecting raw Buffer output without conversion
• Missing the 'end' event message
• Confusing output formatting

4. Identify the error in this code snippet that reads data from a stream:

const fs = require('fs');
const stream = fs.createReadStream('file.txt');
stream.on('data', (chunk) => {
  console.log(chunk);
});
stream.on('finish', () => {
  console.log('Stream finished');
});

medium

A. The 'data' event callback is missing a parameter

B. The stream should be paused before reading

C. The 'finish' event does not exist on Readable streams

D. The file path must be absolute

Solution

1. Step 1: Check event names for Readable streams

   Readable streams emit 'end' when done, not 'finish'. 'finish' is for Writable streams.

2. Step 2: Identify the incorrect event usage

   The code listens for 'finish', which will never fire on a Readable stream, causing the message to never appear.

3. Final Answer:

   The 'finish' event does not exist on Readable streams -> Option C

4. Quick Check:

   Readable streams use 'end', not 'finish' event [OK]

Hint: 'finish' is for writing; use 'end' for reading streams [OK]

Common Mistakes:

• Using 'finish' event on Readable streams
• Forgetting to handle 'end' event
• Assuming 'data' event has no parameters

5. You want to read a large file using a Readable stream and count how many times the word "node" appears, case-insensitive. Which approach correctly handles data chunks to count occurrences without missing matches across chunk boundaries?

hard

A. Convert each chunk to string and count occurrences immediately, ignoring chunk edges

B. Count occurrences in each chunk separately without storing leftover text

C. Use a writable stream instead of a readable stream for counting

D. Concatenate chunks into a string, then count occurrences after 'end' event

Solution

1. Step 1: Understand chunk boundaries in streams

   Chunks may split words, so counting in each chunk separately can miss matches that cross chunk edges.

2. Step 2: Choose a method to avoid missing matches

   Concatenating all chunks into one string and counting after the 'end' event ensures all occurrences are counted correctly.

3. Final Answer:

   Concatenate chunks into a string, then count occurrences after 'end' event -> Option D

4. Quick Check:

   Count after full data read to avoid split word misses [OK]

Hint: Join chunks first, then count words to avoid split matches [OK]

Common Mistakes:

• Counting words in each chunk ignoring splits
• Using writable streams for reading tasks
• Not handling case-insensitive matching