Writing data with Writable streams in Node.js - Performance & Optimization

Performance: Writing data with Writable streams

MEDIUM IMPACT

This concept affects how efficiently data is written to streams, impacting memory usage and responsiveness during I/O operations.

Writing large amounts of data to a Writable stream without handling backpressure

Node.js

const fs = require('fs');
const writable = fs.createWriteStream('output.txt');
function writeData(i) {
  let ok = true;
  while (i < 1e6 && ok) {
    ok = writable.write('Line ' + i + '\n');
    i++;
  }
  if (i < 1e6) {
    writable.once('drain', () => writeData(i));
  } else {
    writable.end();
  }
}
writeData(0);

Checks the return value of write() and waits for 'drain' event to avoid overwhelming the buffer, reducing memory spikes and event loop blocking.

📈 Performance GainPrevents buffer overflow, reduces memory usage spikes, and keeps event loop responsive

Writing large amounts of data to a Writable stream without handling backpressure

Node.js

const fs = require('fs');
const writable = fs.createWriteStream('output.txt');
for (let i = 0; i < 1e6; i++) {
  writable.write('Line ' + i + '\n');
}
writable.end();

Writing data continuously without checking if the stream buffer is full causes high memory usage and can block the event loop.

📉 Performance CostTriggers multiple internal buffer fills causing high memory pressure and potential event loop delays

Performance Comparison

Pattern	Memory Usage	Event Loop Blocking	Backpressure Handling	Verdict
Writing without backpressure check	High (buffers grow large)	Yes (blocks event loop)	No	[X] Bad
Writing with backpressure and drain event	Low (buffers controlled)	No (event loop stays responsive)	Yes	[OK] Good

Rendering Pipeline

Writing data to streams flows through the Node.js event loop and internal buffer management before reaching the OS for actual I/O. Proper backpressure handling ensures smooth data flow without blocking.

→Buffering

→Event Loop

→I/O System Calls

⚠️ BottleneckBuffering stage when internal buffer is full causing write() to return false and blocking further writes

Core Web Vital Affected

INP

This concept affects how efficiently data is written to streams, impacting memory usage and responsiveness during I/O operations.

Optimization Tips

1Always check the return value of writable.write() to detect backpressure.

2Pause writing when write() returns false and wait for the 'drain' event before resuming.

3Avoid writing large amounts of data synchronously to prevent blocking the event loop.

Performance Quiz - 3 Questions

What happens if you write data to a Writable stream without checking the write() return value?

A The data is written instantly without any delay or memory impact.

B The internal buffer may overflow causing high memory use and event loop blocking.

C The stream automatically throttles without developer intervention.

D The stream throws an error immediately.

DevTools: Node.js --inspect with Chrome DevTools Performance panel

How to check: Run your Node.js script with --inspect flag, open Chrome DevTools, record a performance profile during stream writing, and observe event loop delays and memory usage.

What to look for: Look for long event loop blocking times and high memory usage spikes indicating poor backpressure handling.

Practice

1. What is the main purpose of a Writable stream in Node.js?

easy

A. To send data piece by piece to a destination

B. To read data from a file

C. To create a server

D. To handle HTTP requests

Solution

1. Step 1: Understand Writable stream role

   Writable streams are designed to send data to a destination in chunks.

2. Step 2: Compare with other options

   Reading data is done by Readable streams, not Writable. Creating servers and handling HTTP requests are unrelated to Writable streams.

3. Final Answer:

   To send data piece by piece to a destination -> Option A

4. Quick Check:

   Writable stream = send data [OK]

Hint: Writable streams send data out chunk by chunk [OK]

Common Mistakes:

• Confusing Writable with Readable streams
• Thinking Writable streams read data
• Mixing streams with server creation

2. Which of the following is the correct way to implement the _write method in a custom Writable stream?

easy

A. _write(chunk, encoding, callback) { callback(); }

B. _write(chunk, encoding) { return chunk; }

C. _write(chunk) { console.log(chunk); }

D. _write() { return true; }

Solution

1. Step 1: Recall _write method signature

   The _write method must accept three parameters: chunk, encoding, and callback.

2. Step 2: Check callback usage

   Calling callback() signals that the chunk was processed. Omitting it causes the stream to hang.

3. Final Answer:

   _write(chunk, encoding, callback) { callback(); } -> Option A

4. Quick Check:

   _write needs callback() [OK]

Hint: Always include callback in _write and call it [OK]

Common Mistakes:

• Forgetting the callback parameter
• Not calling callback inside _write
• Wrong number of parameters in _write

3. Consider this code snippet:

const { Writable } = require('stream');
class MyStream extends Writable {
  _write(chunk, encoding, callback) {
    console.log(chunk.toString());
    callback();
  }
}
const stream = new MyStream();
stream.write('Hello');
stream.end('World');

What will be printed to the console?

medium

A. HelloWorld

B. Hello\nWorld

C. Hello\nWorld\n

D. Hello\nWorld printed separately

Solution

1. Step 1: Understand write and end calls

   stream.write('Hello') sends 'Hello' chunk, then stream.end('World') sends 'World' chunk and ends.

2. Step 2: Check _write behavior

   Each chunk is logged separately with console.log, so 'Hello' and 'World' print on separate lines.

3. Final Answer:

   Hello World printed separately -> Option D

4. Quick Check:

   Each chunk logs separately [OK]

Hint: Each write chunk logs on its own line [OK]

Common Mistakes:

• Assuming chunks concatenate automatically
• Expecting no newline between chunks
• Confusing write and end data handling

4. What is wrong with this Writable stream implementation?

const { Writable } = require('stream');
class BrokenStream extends Writable {
  _write(chunk, encoding) {
    console.log(chunk.toString());
  }
}
const stream = new BrokenStream();
stream.write('Test');

medium

A. Using console.log inside _write is not allowed

B. Missing callback parameter and not calling callback()

C. Not calling stream.end() causes error

D. _write method should be named write

Solution

1. Step 1: Check _write method signature

   _write must have three parameters: chunk, encoding, callback.

2. Step 2: Check callback usage

   Callback must be called to signal completion; missing callback causes stream to hang.

3. Final Answer:

   Missing callback parameter and not calling callback() -> Option B

4. Quick Check:

   _write needs callback param and call [OK]

Hint: Always include and call callback in _write [OK]

Common Mistakes:

• Omitting callback parameter
• Not calling callback inside _write
• Confusing _write with write method

5. You want to create a Writable stream that collects all written chunks into a single string and logs it only when the stream ends. Which approach is correct?

hard

A. Log each chunk inside _write and ignore 'finish' event

B. Call callback only after all chunks are written, ignoring _write

C. Store chunks in a variable inside _write, call callback, then log in 'finish' event

D. Use readable stream instead of writable for collecting data

Solution

1. Step 1: Collect chunks inside _write

   Inside _write, append each chunk to a variable and call callback to continue.

2. Step 2: Log combined data on 'finish' event

   Listen to the 'finish' event to know when writing ends, then log the full collected string.

3. Final Answer:

   Store chunks in a variable inside _write, call callback, then log in 'finish' event -> Option C

4. Quick Check:

   Collect chunks + log on finish = Store chunks in a variable inside _write, call callback, then log in 'finish' event [OK]

Hint: Collect chunks in _write, log on 'finish' event [OK]

Common Mistakes:

• Logging inside _write causing multiple logs
• Not calling callback causing stream to hang
• Using readable stream instead of writable