Streaming with ReadableStream and WritableStream in Deno

Introduction

Streaming lets you handle data bit by bit instead of all at once. This helps when working with large files or slow networks.

Reading a large file without loading it fully into memory.

Sending data over the network as it becomes available.

Processing live data like video or audio streams.

Transforming data on the fly while copying it.

Saving memory and improving performance with big data.

Syntax

Deno

const readable = new ReadableStream({
  start(controller) {
    // push data chunks with controller.enqueue()
    // call controller.close() when done
  }
});

const writable = new WritableStream({
  write(chunk) {
    // handle each chunk of data
  },
  close() {
    // cleanup when done
  }
});

ReadableStream produces data chunks you can read.

WritableStream accepts data chunks you can write to.

Examples

This creates a readable stream that sends two text chunks then ends.

Deno

const readable = new ReadableStream({
  start(controller) {
    controller.enqueue('Hello');
    controller.enqueue('World');
    controller.close();
  }
});

This writable stream logs each chunk it gets and logs when done.

Deno

const writable = new WritableStream({
  write(chunk) {
    console.log('Received:', chunk);
  },
  close() {
    console.log('Stream closed');
  }
});

This connects the readable stream to the writable stream, sending data automatically.

Deno

readable.pipeTo(writable);

Sample Program

This program creates a readable stream that sends three text chunks. The writable stream logs each chunk it receives and a message when the stream ends. The pipeTo method connects them so data flows automatically.

Deno

const readable = new ReadableStream({
  start(controller) {
    controller.enqueue('Hello ');
    controller.enqueue('Deno ');
    controller.enqueue('Streams!');
    controller.close();
  }
});

const writable = new WritableStream({
  write(chunk) {
    console.log('Chunk received:', chunk);
  },
  close() {
    console.log('All chunks received. Stream closed.');
  }
});

await readable.pipeTo(writable);

Important Notes

Streams help avoid loading all data at once, saving memory.

Use pipeTo() to connect streams easily.

Always close streams to signal no more data.

Summary

ReadableStream produces data chunks you can read.

WritableStream accepts data chunks you can write to.

Use streams to handle large or continuous data efficiently.

Practice

1. What is the main purpose of a ReadableStream in Deno?

easy

A. To accept data chunks that can be written

B. To produce data chunks that can be read

C. To store data permanently

D. To convert data into JSON format

Solution

1. Step 1: Understand the role of ReadableStream

   A ReadableStream is designed to produce data chunks that can be read by consumers.

2. Step 2: Differentiate from WritableStream

   A WritableStream accepts data chunks to be written, which is the opposite role.

3. Final Answer:

   To produce data chunks that can be read -> Option B

4. Quick Check:

   ReadableStream produces data = A [OK]

Hint: ReadableStream produces data; WritableStream consumes data [OK]

Common Mistakes:

• Confusing ReadableStream with WritableStream
• Thinking streams store data permanently
• Assuming streams convert data formats automatically

2. Which of the following is the correct way to create a new WritableStream in Deno?

easy

A. const stream = new WritableStream({ write(chunk) { console.log(chunk); } });

B. const stream = WritableStream.write(chunk => console.log(chunk));

C. const stream = new WritableStream.write(chunk => console.log(chunk));

D. const stream = WritableStream({ write(chunk) { console.log(chunk); } });

Solution

1. Step 1: Recall WritableStream constructor syntax

   WritableStream is created with new WritableStream() and an object with a write method.

2. Step 2: Check each option's syntax

   const stream = new WritableStream({ write(chunk) { console.log(chunk); } }); correctly uses new WritableStream({ write(chunk) { ... } }). Others misuse syntax or omit new.

3. Final Answer:

   const stream = new WritableStream({ write(chunk) { console.log(chunk); } }); -> Option A

4. Quick Check:

   WritableStream needs new + write method object = C [OK]

Hint: Use 'new WritableStream' with an object having a write method [OK]

Common Mistakes:

• Omitting 'new' keyword
• Calling WritableStream as a function
• Passing a function directly instead of an object

3. What will the following code output in Deno?

const readable = new ReadableStream({
  start(controller) {
    controller.enqueue('Hello');
    controller.enqueue('World');
    controller.close();
  }
});

const reader = readable.getReader();

(async () => {
  let result = '';
  while(true) {
    const { done, value } = await reader.read();
    if (done) break;
    result += value + ' ';
  }
  console.log(result.trim());
})();

medium

A. Error: reader.read is not a function

B. HelloWorld

C. Hello World

D. undefined

Solution

1. Step 1: Understand the ReadableStream setup

   The stream enqueues two chunks: 'Hello' and 'World', then closes.

2. Step 2: Analyze the reading loop

   The loop reads each chunk, appends it with a space to result, and stops when done is true.

3. Step 3: Output after trimming

   After reading both chunks, result is 'Hello World ' which trims to 'Hello World'.

4. Final Answer:

   Hello World -> Option C

5. Quick Check:

   Chunks read and concatenated with space = D [OK]

Hint: ReadableStream chunks read sequentially, concatenated with spaces [OK]

Common Mistakes:

• Forgetting to await reader.read()
• Not trimming the final string
• Assuming chunks concatenate without spaces

4. Identify the error in this Deno code using streams:

const writable = new WritableStream({
  write(chunk) {
    console.log(chunk.toUpperCase());
  }
});

const writer = writable.getWriter();

await writer.write(123);
await writer.close();

medium

A. write method must be async

B. Missing await before new WritableStream

C. getWriter() is not a function

D. Cannot call toUpperCase on a number

Solution

1. Step 1: Check the write method usage

   The write method calls chunk.toUpperCase(), which only works on strings.

2. Step 2: Analyze the write call argument

   The code writes the number 123, which has no toUpperCase() method, causing a runtime error.

3. Final Answer:

   Cannot call toUpperCase on a number -> Option D

4. Quick Check:

   toUpperCase on number causes error = A [OK]

Hint: Check data types before calling string methods [OK]

Common Mistakes:

• Assuming write must be async
• Thinking getWriter() is invalid
• Missing await on WritableStream creation

5. You want to transform a ReadableStream of text chunks to uppercase and write them to a WritableStream. Which approach correctly implements this in Deno?

hard

A. Pipe the readable through a TransformStream that uppercases chunks, then pipe to writable

B. Directly write chunks from readable to writable without transformation

C. Use readable.getReader() and writable.getWriter(), but forget to release locks

D. Convert chunks to uppercase after writing to writable

Solution

1. Step 1: Understand stream transformation

   To modify data between readable and writable, use a TransformStream that changes chunks to uppercase.

2. Step 2: Use pipeThrough and pipeTo methods

   Pipe the readable stream through the transform, then pipe the result to the writable stream for correct flow and backpressure handling.

3. Step 3: Evaluate other options

   Directly write chunks from readable to writable without transformation skips transformation, C risks locking issues, and D transforms too late after writing.

4. Final Answer:

   Pipe the readable through a TransformStream that uppercases chunks, then pipe to writable -> Option A

5. Quick Check:

   Use TransformStream + pipeThrough + pipeTo = B [OK]

Hint: Use TransformStream between readable and writable for data changes [OK]

Common Mistakes:

• Skipping TransformStream for data modification
• Not handling stream locks properly
• Transforming data after writing