Buffer overflow attacks in Operating Systems - Step-by-Step Execution

Concept Flow - Buffer overflow attacks

Program allocates buffer

↓

Input data copied into buffer

↓

Is input size > buffer size?

No→Safe execution

Yes↓

Extra data overwrites adjacent memory

↓

Attacker controls overwritten data

↓

Program behavior altered (e.g., execute malicious code)

This flow shows how input larger than a buffer causes extra data to overwrite memory, leading to possible attack.

Execution Sample

Operating Systems

char buffer[6];
strcpy(buffer, input);

Copies input string into a small buffer without size check, risking overflow.

Analysis Table

Step	Input Data	Buffer Content	Memory Overwritten?	Effect
1	"abc"	"abc\0"	No	Normal operation
2	"hello"	"hello\0"	No	Normal operation
3	"helloworld"	"hello"	Yes, extra "world" overwrites	Potential crash or exploit
4	"AAAAAAAAAA"	"AAAAA"	Yes, extra "AAAAA" overwrites	Possible control hijack

💡 Execution stops or behaves unexpectedly when input size exceeds buffer size causing overflow.

State Tracker

Variable	Start	After Step 1	After Step 2	After Step 3	After Step 4
buffer	"\0\0\0\0\0\0"	"abc\0\0\0"	"hello\0"	"hello"	"AAAAA"

Key Insights - 3 Insights

Why does input longer than the buffer size cause problems?

Because the extra input data writes beyond the buffer's allocated space, overwriting adjacent memory as shown in execution_table rows 3 and 4.

Does strcpy check if input fits in the buffer?

No, strcpy blindly copies input, leading to overflow if input is too large, as seen in the execution sample.

What can happen if adjacent memory is overwritten?

The program can crash or attackers can change program behavior, possibly running malicious code, as indicated in the Effect column of the execution_table.

Visual Quiz - 3 Questions

Look at the execution_table, what is the buffer content after step 2?

A "hello\0"

B "abc\0\0"

C "helloworld"

D "AAAAA"

💡 Hint

Check the Buffer Content column for step 2 in the execution_table.

At which step does the input cause memory to be overwritten?

A Step 1

B Step 2

C Step 3

D No step

💡 Hint

Look at the Memory Overwritten? column in the execution_table.

If input was "hi", how would the buffer content change compared to step 1?

A Buffer would be empty

B Buffer would contain "hi\0\0\0\0"

C Buffer would overflow

D Buffer would contain "hello"

💡 Hint

Refer to variable_tracker and how buffer stores input shorter than its size.

Concept Snapshot

Buffer overflow attacks happen when input data is larger than the allocated buffer.
Extra data overwrites adjacent memory, causing crashes or exploits.
Functions like strcpy do not check input size, risking overflow.
Safe coding requires checking input size before copying.
Attackers exploit overflow to run malicious code or hijack control.

Full Transcript

Buffer overflow attacks occur when a program copies more data into a buffer than it can hold. This extra data overwrites nearby memory, which can cause the program to crash or behave unexpectedly. For example, using strcpy to copy input into a small buffer without checking size can cause overflow. The execution table shows how inputs of different lengths affect the buffer and memory. When input fits, the program runs normally. When input is too large, memory is overwritten, leading to potential security risks. Understanding this helps programmers write safer code by validating input sizes.

Practice

1. What is a buffer overflow attack in simple terms?

easy

A. Putting more data into a memory space than it can hold

B. Deleting data from memory intentionally

C. Encrypting data to protect it

D. Backing up data to a safe location

Solution

1. Step 1: Understand what a buffer is

   A buffer is a small area in memory used to store data temporarily.

2. Step 2: Define buffer overflow

   Buffer overflow happens when more data is put into this area than it can hold, causing errors or security risks.

3. Final Answer:

   Putting more data into a memory space than it can hold -> Option A

4. Quick Check:

   Buffer overflow = Too much data in small memory [OK]

Hint: Too much data in small space causes overflow [OK]

Common Mistakes:

• Confusing overflow with data deletion
• Thinking encryption is overflow
• Mixing backup with overflow

2. Which of the following is the correct way to prevent buffer overflow in programming?

easy

A. Using unlimited memory without checks

B. Ignoring input size and trusting user data

C. Checking input size before storing it in memory

D. Allowing programs to crash to fix errors

Solution

1. Step 1: Identify safe programming practice

   Safe programming means checking input size before storing it to avoid overflow.

2. Step 2: Evaluate options

   Ignoring input or using unlimited memory without checks is unsafe; allowing crashes is not prevention.

3. Final Answer:

   Checking input size before storing it in memory -> Option C

4. Quick Check:

   Input size check = Prevent overflow [OK]

Hint: Always check input size before storing [OK]

Common Mistakes:

• Trusting user input blindly
• Assuming memory is unlimited
• Ignoring program crashes

3. Consider this code snippet in C-like pseudocode:

char buffer[5];
strcpy(buffer, "HelloWorld");

What will likely happen when this code runs?

medium

A. The program stores "Hello" safely and ignores the rest

B. The program crashes or behaves unpredictably due to overflow

C. The program automatically resizes buffer to fit the string

D. The program throws a syntax error before running

Solution

1. Step 1: Analyze buffer size and input

   The buffer can hold 5 characters, but the string "HelloWorld" has 10 characters.

2. Step 2: Understand strcpy behavior

   strcpy copies all characters without checking size, causing overflow beyond buffer limits.

3. Final Answer:

   The program crashes or behaves unpredictably due to overflow -> Option B

4. Quick Check:

   Copying too large string = Crash or error [OK]

Hint: Copying bigger string than buffer causes crash [OK]

Common Mistakes:

• Assuming partial copy happens safely
• Thinking buffer resizes automatically
• Confusing syntax error with runtime error

4. Identify the error in this pseudocode that causes a buffer overflow:

char data[4];
input = getUserInput();
for i in range(len(input)) {
  data[i] = input[i];
}

medium

A. The data array is declared with too many elements

B. The input is not converted to uppercase

C. The loop uses the wrong variable name

D. The loop does not check if i exceeds data size

Solution

1. Step 1: Examine loop and array size

   The data array has size 4, but the loop runs for the full length of input without limit.

2. Step 2: Identify missing boundary check

   Without checking if i < 4, the loop writes beyond data's memory, causing overflow.

3. Final Answer:

   The loop does not check if i exceeds data size -> Option D

4. Quick Check:

   Missing boundary check = Overflow risk [OK]

Hint: Always check loop limits against buffer size [OK]

Common Mistakes:

• Ignoring array size limits
• Focusing on unrelated input format
• Confusing variable names

5. A program uses a buffer of size 8 to store user input. Which approach best prevents buffer overflow while allowing flexible input length?

hard

A. Use dynamic memory allocation to resize buffer as needed

B. Use a fixed buffer and truncate input longer than 8 characters

C. Ignore input size and trust user to enter short data

D. Allow buffer overflow and handle errors after crash

Solution

1. Step 1: Understand fixed buffer limits

   Fixed buffer size limits input length, truncation loses data, ignoring size risks overflow.

2. Step 2: Evaluate dynamic allocation benefits

   Dynamic memory allocation adjusts buffer size to input length, preventing overflow and preserving data.

3. Final Answer:

   Use dynamic memory allocation to resize buffer as needed -> Option A

4. Quick Check:

   Dynamic resizing = Safe flexible input [OK]

Hint: Resize buffer dynamically to fit input safely [OK]

Common Mistakes:

• Relying on truncation losing data
• Trusting user input blindly
• Ignoring errors after overflow