Buffer overflow attacks in Operating Systems - Step-by-Step Execution

Concept Flow - Buffer overflow attacks

Program allocates buffer

↓

Input data copied into buffer

↓

Is input size > buffer size?

No→Safe execution

Yes↓

Extra data overwrites adjacent memory

↓

Attacker controls overwritten data

↓

Program behavior altered (e.g., execute malicious code)

This flow shows how input larger than a buffer causes extra data to overwrite memory, leading to possible attack.

Execution Sample

Operating Systems

char buffer[6];
strcpy(buffer, input);

Copies input string into a small buffer without size check, risking overflow.

Analysis Table

Step	Input Data	Buffer Content	Memory Overwritten?	Effect
1	"abc"	"abc\0"	No	Normal operation
2	"hello"	"hello\0"	No	Normal operation
3	"helloworld"	"hello"	Yes, extra "world" overwrites	Potential crash or exploit
4	"AAAAAAAAAA"	"AAAAA"	Yes, extra "AAAAA" overwrites	Possible control hijack

💡 Execution stops or behaves unexpectedly when input size exceeds buffer size causing overflow.

State Tracker

Variable	Start	After Step 1	After Step 2	After Step 3	After Step 4
buffer	"\0\0\0\0\0\0"	"abc\0\0\0"	"hello\0"	"hello"	"AAAAA"

Key Insights - 3 Insights

Why does input longer than the buffer size cause problems?

Because the extra input data writes beyond the buffer's allocated space, overwriting adjacent memory as shown in execution_table rows 3 and 4.

Does strcpy check if input fits in the buffer?

No, strcpy blindly copies input, leading to overflow if input is too large, as seen in the execution sample.

What can happen if adjacent memory is overwritten?

The program can crash or attackers can change program behavior, possibly running malicious code, as indicated in the Effect column of the execution_table.

Visual Quiz - 3 Questions

Look at the execution_table, what is the buffer content after step 2?

A "hello\0"

B "abc\0\0"

C "helloworld"

D "AAAAA"

💡 Hint

Check the Buffer Content column for step 2 in the execution_table.

At which step does the input cause memory to be overwritten?

A Step 1

B Step 2

C Step 3

D No step

💡 Hint

Look at the Memory Overwritten? column in the execution_table.

If input was "hi", how would the buffer content change compared to step 1?

A Buffer would be empty

B Buffer would contain "hi\0\0\0\0"

C Buffer would overflow

D Buffer would contain "hello"

💡 Hint

Refer to variable_tracker and how buffer stores input shorter than its size.

Concept Snapshot

Buffer overflow attacks happen when input data is larger than the allocated buffer.
Extra data overwrites adjacent memory, causing crashes or exploits.
Functions like strcpy do not check input size, risking overflow.
Safe coding requires checking input size before copying.
Attackers exploit overflow to run malicious code or hijack control.

Full Transcript

Buffer overflow attacks occur when a program copies more data into a buffer than it can hold. This extra data overwrites nearby memory, which can cause the program to crash or behave unexpectedly. For example, using strcpy to copy input into a small buffer without checking size can cause overflow. The execution table shows how inputs of different lengths affect the buffer and memory. When input fits, the program runs normally. When input is too large, memory is overwritten, leading to potential security risks. Understanding this helps programmers write safer code by validating input sizes.