Stream backpressure concept in Node.js - Step-by-Step Execution

Concept Flow - Stream backpressure concept

Readable Stream pushes data

↓

Writable Stream buffer fills

↓

Buffer reaches highWaterMark?

No→Continue pushing

Yes↓

Pause Readable Stream

↓

Writable Stream drains buffer

↓

Resume Readable Stream

↓

Repeat cycle until end

Data flows from readable to writable streams. When writable buffer is full, readable pauses to prevent overload, then resumes when ready.

Execution Sample

Node.js

const readable = getReadableStream();
const writable = getWritableStream();

readable.on('data', chunk => {
  if (!writable.write(chunk)) {
    readable.pause();
  }
});

writable.on('drain', () => {
  readable.resume();
});

This code shows how a readable stream pauses when writable buffer is full and resumes when drained.

Execution Table

Step	Readable Event	Writable write() returns	Action on Readable	Writable Buffer State
1	data chunk1	true	continue	buffer has chunk1
2	data chunk2	true	continue	buffer has chunk1, chunk2
3	data chunk3	false	pause readable	buffer full with chunk1, chunk2, chunk3
4	no data (paused)	N/A	paused	buffer full
5	drain event	N/A	resume readable	buffer emptied
6	data chunk4	true	continue	buffer has chunk4
7	end event	N/A	stop reading	buffer processed
8	no more data	N/A	end	empty buffer

💡 Readable stream ends and writable buffer is empty, so data flow stops.

Variable Tracker

Variable	Start	After Step 1	After Step 2	After Step 3	After Step 5	After Step 6	Final
readablePaused	false	false	false	true	false	false	false
writableBufferFull	false	false	false	true	false	false	false

Key Moments - 3 Insights

Why does the readable stream pause when writable.write() returns false?

Because writable.write() returning false means the writable buffer is full (see step 3 in execution_table), so pausing readable prevents data overflow.

When does the readable stream resume after pausing?

It resumes on the writable stream's 'drain' event (step 5), indicating the buffer has emptied enough to accept more data.

What happens if we never listen to the 'drain' event?

The readable stream would stay paused indefinitely after writable buffer fills, stopping data flow (see step 4).

Visual Quiz - 3 Questions

Look at the execution_table, what is the value of readablePaused after step 3?

A false

B true

C undefined

D null

💡 Hint

Check the variable_tracker row for readablePaused at After Step 3.

At which step does the writable buffer become full, causing readable to pause?

A Step 3

B Step 5

C Step 1

D Step 7

💡 Hint

Look at execution_table column 'Writable write() returns' and 'Action on Readable'.

If the 'drain' event listener is removed, what happens to the readable stream after step 3?

A It throws an error

B It resumes normally

C It pauses indefinitely

D It ends immediately

💡 Hint

Refer to key_moments question about missing 'drain' event listener.

Concept Snapshot

Stream backpressure controls data flow between streams.
Writable stream signals full buffer by returning false on write().
Readable stream pauses to avoid overload.
Readable resumes on writable 'drain' event.
Prevents memory overflow and keeps data flow smooth.

Full Transcript

Stream backpressure is a way Node.js streams manage data flow to avoid overload. When a readable stream sends data to a writable stream, the writable stream may get full. It tells the readable stream by returning false from its write() method. Then, the readable stream pauses sending more data. When the writable stream empties its buffer, it emits a 'drain' event. The readable stream listens for this event and resumes sending data. This cycle repeats until all data is processed. This mechanism prevents memory overflow and keeps streams working efficiently.

Practice

1. What is the main purpose of backpressure in Node.js streams?

easy

A. To speed up data transfer between streams

B. To control the flow of data and prevent writable streams from being overwhelmed

C. To close streams automatically after data transfer

D. To convert data formats between streams

Solution

1. Step 1: Understand stream data flow

   Streams send data from readable to writable. If writable is slow, data can pile up.

2. Step 2: Role of backpressure

   Backpressure pauses the readable stream to avoid overwhelming the writable stream.

3. Final Answer:

   To control the flow of data and prevent writable streams from being overwhelmed -> Option B

4. Quick Check:

   Backpressure controls flow = A [OK]

Hint: Backpressure means controlling data flow to avoid overload [OK]

Common Mistakes:

• Thinking backpressure speeds up data
• Confusing backpressure with stream closing
• Assuming backpressure changes data format

2. Which of the following is the correct way to listen for the 'drain' event on a writable stream in Node.js?

easy

A. writable.on('drain', () => { /* handle drain */ });

B. writable.emit('drain', () => { /* handle drain */ });

C. writable.listen('drain', () => { /* handle drain */ });

D. writable.addEventListener('drain', () => { /* handle drain */ });

Solution

1. Step 1: Recall event listening syntax in Node.js streams

   Streams use the .on() method to listen for events.

2. Step 2: Identify correct method for 'drain' event

   The 'drain' event is listened to with writable.on('drain', callback).

3. Final Answer:

   writable.on('drain', () => { /* handle drain */ }); -> Option A

4. Quick Check:

   Use .on() to listen to events = D [OK]

Hint: Use .on() to listen for stream events like 'drain' [OK]

Common Mistakes:

• Using emit() instead of on() to listen
• Using browser event methods like addEventListener
• Using non-existent listen() method

3. Consider this code snippet:

const readable = getReadableStreamSomehow();
const writable = getWritableStreamSomehow();

readable.on('data', chunk => {
  const canWrite = writable.write(chunk);
  if (!canWrite) {
    readable.pause();
  }
});

writable.on('drain', () => {
  readable.resume();
});

readable.on('end', () => {
  writable.end();
});

What will happen if the writable stream's internal buffer is full?

medium

A. The writable stream will discard new data chunks

B. The readable stream will continue sending data without pause

C. The program will throw an error and crash

D. The readable stream will pause until the writable stream drains

Solution

1. Step 1: Analyze writable.write() return value

   When writable.write() returns false, it means the buffer is full and cannot accept more data now.

2. Step 2: Check how readable reacts

   On false, readable.pause() is called to stop sending data temporarily.

3. Step 3: Understand 'drain' event handling

   When writable drains, it emits 'drain', triggering readable.resume() to continue data flow.

4. Final Answer:

   The readable stream will pause until the writable stream drains -> Option D

5. Quick Check:

   Writable full -> readable pauses -> resumes on drain = C [OK]

Hint: Writable.write false means pause readable until drain event [OK]

Common Mistakes:

• Assuming readable never pauses
• Thinking writable discards data silently
• Expecting program crash on full buffer

4. You wrote this code to handle backpressure but the readable stream never resumes after pausing:

readable.on('data', chunk => {
  if (!writable.write(chunk)) {
    readable.pause();
  }
});

// Missing 'drain' event listener on writable

readable.on('end', () => {
  writable.end();
});

What is the main problem causing the readable stream to stay paused?

medium

A. The 'drain' event listener is missing, so readable never resumes

B. The writable stream should not use write() inside 'data' event

C. The readable stream should call end() instead of pause()

D. The 'end' event should be listened on writable, not readable

Solution

1. Step 1: Identify missing event listener

   The code pauses readable when writable.write returns false but never listens for 'drain'.

2. Step 2: Understand consequence of missing 'drain'

   Without 'drain' listener calling readable.resume(), readable stays paused indefinitely.

3. Final Answer:

   The 'drain' event listener is missing, so readable never resumes -> Option A

4. Quick Check:

   Missing drain listener -> readable stuck paused = B [OK]

Hint: Always listen for 'drain' to resume paused readable streams [OK]

Common Mistakes:

• Thinking pause() should be replaced by end()
• Ignoring the need for 'drain' event
• Confusing 'end' event on readable vs writable

5. You want to implement backpressure handling manually without using pipe(). Which sequence correctly manages backpressure between a readable and writable stream?

hard

A. On 'data', pause readable; write chunk; on writable 'error', resume readable

B. On 'data', write chunk; always resume readable immediately; on writable 'finish', pause readable

C. On 'data', write chunk; if write returns false, pause readable; on writable 'drain', resume readable

D. On 'data', write chunk; if write returns true, pause readable; on writable 'close', resume readable

Solution

1. Step 1: Understand backpressure manual handling

   When writable.write returns false, it signals buffer full, so readable must pause.

2. Step 2: Resume readable on 'drain' event

   Writable emits 'drain' when ready for more data, so readable resumes then.

3. Step 3: Verify option correctness

   On 'data', write chunk; if write returns false, pause readable; on writable 'drain', resume readable matches this correct sequence; others misuse pause/resume or wrong events.

4. Final Answer:

   On 'data', write chunk; if write returns false, pause readable; on writable 'drain', resume readable -> Option C

5. Quick Check:

   Pause on false write, resume on drain = A [OK]

Hint: Pause readable if write returns false; resume on writable 'drain' [OK]

Common Mistakes:

• Resuming readable immediately without pause
• Pausing readable on true write return
• Using wrong events like 'finish' or 'close' for resume