My Simple C Program Demonstrating Buffer Overflow

Note

This is the eleventh assignment from my Masters Advanced Network Security Course which has never been published anywhere and I, as the author and copyright holder, license this assignment customized CC-BY-SA where anyone can share, copy, republish, and sell on condition to state my name as the author and notify that the original and open version available here.

1. Introduction

A buffer overflow contains the word buffer which is a temporary data storage area and overflow where too much data is inserted it overflows. Buffer overflow can be define as an event where the data size that was input exceeds the size of the buffer and the last part of the data overwrite the other section of the memory [1]. Think of it as a tea spills out of the cup if we put too much tea in it, and same when the data is too much that it will spill out.

Different from a tea being spilled buffer overflow is a data that may overwrites other parts of the memory and maybe read and executed by the machine. The spilled code could change, damage, add, or delete the user’s data, even worst may contain an program to execute a remote connection to a malicious person. In July 2000 a buffer overflow vulnerability was found in Microsoft Outlook. No need for a virus attachment, just receiving an email with an exceeded header size added with a payload already open a session for the attacker. These makes buffer overflow as a famous security attack. This attack began when C supplied the framework and poor programming was practice [1].

2. Buffer Overflow in Security

On Figure 1 shows memory layout of Linux process. A process is a program in execution, and an executable program contains a set of binary instructions to be executed on the processor. They could be read only data like printf, global or static data that last through out the process, brk pointer that keeps track of malloced memory, or local function that cleans up after it is run [2].

Figure 1. Linux Memory Layout

A process image on Figure 1 shows that it started with the program’s code and data on the first 2 blocks after the unused block. Following is the runtime heap created at runtime by malloc, then goes to…