Buffer allocation and encoding in Node.js - Step-by-Step Execution

Concept Flow - Buffer allocation and encoding

Start

↓

Allocate Buffer

↓

Write Data with Encoding

↓

Read or Use Buffer

↓

End

This flow shows how Node.js allocates a buffer, writes data with a specific encoding, then reads or uses the buffer.

Execution Sample

Node.js

const buf = Buffer.alloc(5);
buf.write('Hi', 'utf8');
console.log(buf);
console.log(buf.toString('utf8'));

Allocates a 5-byte buffer, writes 'Hi' in UTF-8 encoding, then prints the buffer and its string form.

Execution Table

Step	Action	Buffer Content (hex)	Buffer Content (string)	Notes
1	Allocate buffer of length 5	00 00 00 00 00		Buffer initialized with zeros
2	Write 'Hi' with utf8 encoding	48 69 00 00 00	Hi	'H' = 0x48, 'i' = 0x69, rest zeros
3	Print buffer object	<Buffer 48 69 00 00 00>		Shows raw bytes in hex
4	Convert buffer to string utf8	48 69 00 00 00	Hi	Decodes entire buffer; null bytes (\u0000) included but non-printable in console
5	End			Process complete

💡 Buffer allocated fixed size; write fills bytes; toString decodes entire buffer by default (null bytes included)

Variable Tracker

Variable	Start	After Step 1	After Step 2	After Step 4	Final
buf	undefined	<Buffer 00 00 00 00 00>	<Buffer 48 69 00 00 00>	<Buffer 48 69 00 00 00>	<Buffer 48 69 00 00 00>

Key Moments - 3 Insights

Why does the buffer show zeros after the written characters?

Because the buffer was allocated with a fixed size of 5 bytes, and only the first 2 bytes were written with 'Hi'. The remaining 3 bytes remain zero as shown in step 2 of the execution table.

Why does buf.toString('utf8') output 'Hi' and not include the trailing zeros?

The toString method decodes the entire buffer as UTF-8, resulting in 'Hi\u0000\u0000\u0000'. Null bytes (\u0000) are non-printable, so they do not visibly appear in console output (step 4).

What happens if you write a string longer than the buffer size?

Only the bytes that fit in the allocated buffer are written; the rest are ignored. This prevents buffer overflow but truncates the string.

Visual Quiz - 3 Questions

Look at the execution table, what is the buffer content in hex after writing 'Hi'?

A 00 00 00 00 00

B 48 69 00 00 00

C 48 69 48 69 00

D 00 48 69 00 00

💡 Hint

Check step 2 in the execution table for buffer content after write

At which step does the buffer get allocated with zeros?

A Step 2

B Step 3

C Step 1

D Step 4

💡 Hint

Look at the first step where the buffer is created

If you write 'HelloWorld' into the 5-byte buffer, what happens?

A Buffer contains 'Hello' and ignores the rest

B Buffer expands automatically to fit all

C Buffer throws an error

D Buffer contains 'World'

💡 Hint

Recall buffer size is fixed at allocation (step 1) and write truncates excess

Concept Snapshot

Buffer.alloc(size) creates a fixed-size buffer filled with zeros.
Use buf.write(string, encoding) to write data into the buffer.
Writing beyond buffer size truncates the string.
buf.toString(encoding) decodes the entire buffer as string by default; null bytes become \u0000.
Buffers hold raw bytes, useful for binary data handling in Node.js.

Full Transcript

This lesson shows how Node.js buffers are allocated and used. First, a buffer of fixed size is created with Buffer.alloc, filled with zeros. Then, writing a string like 'Hi' with UTF-8 encoding fills the first bytes of the buffer. The rest remain zero. When printing the buffer, you see the raw bytes in hex. Converting the buffer to a string decodes all bytes; trailing null bytes are included but invisible when printed. Writing a string longer than the buffer size only writes what fits, truncating the rest. This helps manage binary data safely and efficiently.

Practice

1. What does Buffer.alloc(5) do in Node.js?

easy

A. Creates a buffer of length 5 filled with zeros

B. Creates a buffer of length 5 filled with random data

C. Creates a buffer from a string of length 5

D. Allocates memory but does not initialize the buffer

Solution

1. Step 1: Understand Buffer.alloc behavior

   Buffer.alloc(size) creates a buffer of the given size and fills it with zeros for safety.

2. Step 2: Apply to size 5

   Calling Buffer.alloc(5) creates a buffer of length 5 filled with zeros.

3. Final Answer:

   Creates a buffer of length 5 filled with zeros -> Option A

4. Quick Check:

   Buffer.alloc(5) = zero-filled buffer [OK]

Hint: Buffer.alloc always zero-fills the buffer size [OK]

Common Mistakes:

• Thinking Buffer.alloc fills with random data
• Confusing Buffer.alloc with Buffer.from
• Assuming buffer is uninitialized

2. Which of the following is the correct syntax to create a buffer from the string 'hello' using UTF-8 encoding?

easy

A. Buffer.alloc('hello', 'utf8')

B. Buffer.create('hello', 'utf8')

C. new Buffer('hello', 'utf8')

D. Buffer.from('hello', 'utf8')

Solution

1. Step 1: Identify correct Buffer creation method

   In modern Node.js, Buffer.from() creates a buffer from a string with encoding.

2. Step 2: Check syntax correctness

   Buffer.from('hello', 'utf8') is correct syntax; Buffer.alloc does not accept string input, and Buffer.create does not exist. new Buffer() is deprecated.

3. Final Answer:

   Buffer.from('hello', 'utf8') -> Option D

4. Quick Check:

   Use Buffer.from for string buffers [OK]

Hint: Use Buffer.from for strings, Buffer.alloc for size [OK]

Common Mistakes:

• Using Buffer.alloc with string input
• Using deprecated new Buffer()
• Using non-existent Buffer.create()

3. What will be the output of the following code?

const buf = Buffer.from('abc', 'utf8');
console.log(buf.length);

medium

A. 3

B. 6

C. 1

D. Error

Solution

1. Step 1: Understand Buffer.from with UTF-8 string

   The string 'abc' has 3 characters, each encoded as 1 byte in UTF-8.

2. Step 2: Check buffer length property

   The buffer length equals the number of bytes, so buf.length is 3.

3. Final Answer:

   3 -> Option A

4. Quick Check:

   UTF-8 'abc' length = 3 bytes [OK]

Hint: Buffer length equals byte count of string encoding [OK]

Common Mistakes:

• Assuming length is character count for multi-byte chars
• Expecting length to be string length property
• Confusing bytes with characters

4. Identify the error in this code snippet:

const buf = Buffer.alloc(5, 'abc');
console.log(buf.toString());

medium

A. Buffer.alloc cannot take a string as fill

B. toString() is not a Buffer method

C. No error, prints 'abcab'

D. Buffer.alloc size must be a string

Solution

1. Step 1: Check Buffer.alloc parameters

   Buffer.alloc(size, fill) accepts a size and a fill value. The fill can be a string, which repeats to fill the buffer.

2. Step 2: Understand fill behavior and toString()

   Filling 5 bytes with 'abc' repeats 'abcab'. toString() converts buffer back to string.

3. Final Answer:

   No error, prints 'abcab' -> Option C

4. Quick Check:

   Buffer.alloc fills string repeatedly [OK]

Hint: Buffer.alloc fills string repeatedly if fill is string [OK]

Common Mistakes:

• Thinking Buffer.alloc fill cannot be string
• Assuming toString() is invalid on Buffer
• Expecting error due to fill length mismatch

5. You want to create a buffer from the string 'café' and then convert it back to a string. Which encoding should you use to preserve the accented character correctly?

hard

A. 'ascii'

B. 'utf8'

C. 'base64'

D. 'hex'

Solution

1. Step 1: Understand character encoding for accented characters

   ASCII encoding cannot represent accented characters like 'é'. UTF-8 supports all Unicode characters including accented ones.

2. Step 2: Choose encoding for buffer creation and conversion

   Using 'utf8' ensures the accented character is preserved when converting to and from buffer.

3. Final Answer:

   'utf8' -> Option B

4. Quick Check:

   Use UTF-8 for accented characters [OK]

Hint: Use UTF-8 to handle accented characters correctly [OK]

Common Mistakes:

• Using ASCII which drops accents
• Using base64 or hex which are encoding formats, not text encodings
• Confusing base64 with UTF-8