Buffer Overflow Attack & Defense

Abstract

This paper attempts to explain one of the critical buffer overflow vulnerabilities and its detection approaches that check the referenced buffers at run time, moreover suggesting other protection mechanics applied during software deployment configuration. Programs typically written in C or C++ language are inherently susceptible to buffer overflow attacks, in which methods are often passed pointers or arrays as parameters without any indication of their size, and such malpractices are exploited later. Buffer overflows remain one of the most critical threats to systems security, especially for deployed software. Successful mistreatment of a buffer overflow attack often leads to arbitrary code execution in the form of so-called shell code, and thorough control of the vulnerable application in a vicious manner.

Become a certified reverse engineer!

Get live, hands-on malware analysis training from anywhere, and become a Certified Reverse Engineering Analyst.

Start Learning

Essentials

We shall showcase buffer overflow vulnerability in a Windows environment via C++ or VC++ code which is typically written via VS 2o1o or Turbo C++. Moreover, it is expected that researchers have a comprehensive understanding about C++ syntax and concepts, especially pointers and arrays by creating a Win32 console application.

• Turbo C++ compiler
• VC++.NET
• GCC Compiler (optional)

[download]

Buffer Overflow Bug Demo

An overflow typically happens when something is filled beyond its capacity. So, buffer overrun attacks obviously occur in any program execution that allows input to be written beyond the end of an assigned buffer (memory block). Thus, it leads the data to overwrite into adjacent memory locations which are already occupied to some existing code instruction. In buffer overflow attacks, the hacker encroaches the preoccupied memory segments for other operation instruction sets to inject malicious arbitrary code, and the pre-determined program behavior is changed eventually. These buffer overflows are the implication of poor programming practices by not putting any boundaries on the size of input the program can handle. C and C++ programmed code are a great source to produce buffer overflow attacks, because these languages allow direct access to application memory.

Sometimes hackers find other ways to exploit the overflow besides getting their code to run. Certain overflows do not actually allow hackers to take control, but might instead allow them to manipulate extra data. Let's examine the following bofVul.exe login console based program which accepts user name and password at the command line to validate users. If they enter the correct username and password, it allows access; otherwise, access is denied as follows:

This program was running perfectly up till now, but now imagine if a person with a vicious intention enters the parameters in the following form. He is trying to overflow the buffer by entering some garbage values and finally notices that we successfully penetrate the program even without having the correct user name and password. Bingo!!!!!!!!! It is even revealing the welcome message which is flashed when the user enters the correct credentials. So, this is a bit strange, how can this be possible? We have just entered a sequence of raw data in spite of the password and successfully obtained access.

Using a different password with the same user id still worked! So it is a clear case of a buffer overflow bug because the strange behavior of program allows you to log in if you specify a long password, regardless of whether the password is correct.

Buffer Overrun Internal

A buffer overflow is one of the costliest security vulnerabilities known to affect computer software. It is basically defined as when input is larger than the space allocated for it, but it is written there anyhow and memory is overwritten outside the allocated location. In some cases, overflows result from incorrect handling of mathematical operation or attempts to use memory after the memory has already been allocated. Although many overflows occur when the program receives more data than it expects, in fact there are many different kinds of overflows. It is important to distinguish between various classes of overflows to be able to develop good test cases to identify specific types of overflows.

• Integer overflow: When a specific data type of CPU register meant to hold values within a certain range is assigned a value outside that range. An integer overflow often leads to a buffer overflow in cases in which integer overflow occurs when computing the size of the memory to allocate.
• Stack Overflows: Such overflows occur when data is written past the end of buffers allocated on the stack.
• Heap Overflow: It occurs when data is written outside the space that was allocated for it on the heap.
• Format String Attacks: Format string attacks occur when the %n parameter of the format string is used to write data outside the target buffer.
  It is important to delve deep into the CPU internal infrastructure by examining various registers which play a significant role in memory allocation.
• EIP [Extended Instructor Pointer]: It is only administrated by the CPU and determines next-to-execute opcode in the memory. It contains the offsets of data and instructions.
• ESP [Extended Stack Pointer]: It points to the zenith of the stack to assist the CPU to perform a push and pop operation.
• EBP [Extended Base Pointer]: It is used as a reference point for indirect addressing.
• EAX/EBX/ECX/EDX: They are used for arithmetic and data movement.
• Segments

  [CS/DS/SS/FS/ES/GS]:

  They are used as a base location for program data, instruction and stack.

If a method is called by assembler 'call' commands, a new stackframe is created, with boundaries defined by the EBP and ESP. First, the call command pushes the EIP into the stack to start execution. The previous ESP becomes the new EBP and then space for variables is allocated by subtracting its size from the earlier ESP. Finally, at the end of the function call, the ESP becomes the new EBP.

Now, let's consider one more buffer overflow samples which are developed under VC++ Studio. Here the user name and password are supplied as a command line argument which is copied into a corresponding fixed length array of character variables by using the strcpy method. Later, the supplied credentials are validated against a predefined password via the strcmp method as follows:

[c]
#define BUFF_SIZE 10

void creed(char *usr,char *password)

{

char uN[10];

strcpy(uN, usr);
strcpy(pass, password);

if(strcmp(pass,"ajay"))

{

printf ("n Access Denied n");

}

else

{

printf ("n Welcome:");

..

}

int main(int argc, char* argv[])

{

..

creed(argv[1],argv[2]);

return 0;

}

The moment a user enters tom as a user name and ajay as a password via the command line argument, this program successfully validates those credentials and allows access as follows:

Now try to enter some bogus data as credentials. As assumed, the program won't allow us to get access as follows. At this moment, everything is running fine and under control.

The character variable uN and pass can hold only up to 10 characters and if we input data beyond this fixed length, since we are not performing any bound checking, we are just directly copying the entered data into the buffer directly via the strcpy method. The program would be confused and can't handle such abundant data, which later leads to buffer overflow as follows:

Since we are testing this program under the Windows environment, the OS throws the aforesaid exception, which eventually causes the application to crash, because the program accepted too much data beyond the limit of 10. In the case of compiling this program via Turbo compiler, it notifies the buffer overflow exception in a different manner as follows:

When executing the aforesaid code, it first pushes the two arguments (user name and password) to creed() method backwards onto the stack. It then calls the creed() function. The instruction CALL then pushes the instruction pointer (EIP) onto the stack. The creed () function now pushes the stack frame pointer onto the stack. The current stack pointer (ESP) is then copied into the EBP, making it the new frame pointer (SFP) as follows:

Now, the creed() function instruction next instruction address 0x00412206 is saved to the stack, and execution jumps to ebp in the creed() instruction code where user name and password values are copied into eax, which are pushed into stack. Finally, on behalf of both strcpy offsets, the strcmp instruction is executed.

Thereafter, the ret opcode is executed, which points out the end of program instructions. If a parameter is entered in the correct form or lesser than the fixed length, the program doesn't show any abnormal behavior. But as we are passing the argument beyond the limit, here we examine the register EBP value as 79797979 which becomes the ESP now as follows:

As we move ahead, the execution should jump to 00412209 instead of 0079797979. Hence, Visual Studio throws a run time exception at 79797979 offset where the program denies reading the address space at 79797979 locations. So, the program crashes because execution is halted due to access violation, and buffer overflow attacks occur as follows:

Protection Mechanisms

The buffer overrun attacks can be thwarted in the Windows environment by making critical configuration changes. Visual Studio C++ compiler offers several options to enable certain checks at runtime such as /GS, RTC, Runtime library check and DEP. These options can be enabled using a specific compiler flag. The /GS option shield against vulnerable parameters passes into a function in the form of a pointer, string buffer, or C++ reference. Normally, the incoming methods parameters are assigned on the stack and are susceptible to being overwritten, just like the return address. To avoid this situation, the compiler makes a replica of the vulnerable incoming parameters after storage for local buffers, where they are not in threat of being overwritten.

On the other side, the RTC compiler option control run-time checks such as underflow and overflow checking, stack verification and detection of variable use without initialization. However, these run-time checks introduce a performance overhead that is not acceptable for release builds. We must to enable these compiler checks at least:

• Buffer Security check (/GS)
• Runtime Library check (Both /RTC1…)
• Basic Runtime checks (Enable VC++ Run time Library)
• DEP

Visual Studio also provides a Data Execution Prevention (DEP) option during compilation in case of not disabling it at the operating system level.

Data Execution Prevention (DEP) is an important feature to protect from buffer overflow attacks. This feature has been available on Windows and assumes that no code is intended to be executed that is not part of the program itself. It uses NX technology to prevent the execution of instructions stored in data segments. This feature requires administrative right to change its settings. We can alter this configuration from the command prompt as follows:

For Disable Data Execution Protection Setting

bcdedit.exe /set {current} nx AlwaysOff

For Enabling Data Execution Protection Setting

bcdedit.exe /set {current} nx AlwaysOn

We can enable this setting from My Computer advanced setting under Performance options. These options are disabled by default. In order to enable them, log in via Administrative account as follows:

After finishing with all the necessary configuration or BOF attack thwarting option enabling, run the program and supply some bogus argument beyond the buffer limit. The operating system will issue a run time buffer overflow exception as follows:

Even though /GS aborted the program, these overruns should be fixed. Buffer overflow attacks can be avoided at the time of coding by ensuring that input data does not exceed the size of the fixed length buffer in which it is stored. Here, the fixed length buffer size is 10, so calculate the entered data length and make sure it is less than 10 as follows:

[c]
#define BUFF_SIZE 10

void creed(char *usr, char *password)

{

..

if (strlen(password)<BUFF_SIZE)

{

strcpy(uN, usr);

strcpy(pass, password);

}

else

{

printf ("n Program doesn't support this password n");

exit(1);

}

...

}

int main(int argc, char* argv[])

{

..

creed(argv[1],argv[2]);

return 0;

}

Now a buffer overflow attack can be thwarted even if other protections such GS and DEP are not applied at solution configuration. Here, the program alters and exits if data is entered beyond the buffer limit as follows:

As we have stated earlier, C and C++ sources are most vulnerable to buffer overrun attack. I am going to pinpoint some C library methods which make you vulnerable. Hence, it is recommended to avoid using these methods into your source code.

Final Note

Become a certified reverse engineer!

Get live, hands-on malware analysis training from anywhere, and become a Certified Reverse Engineering Analyst.

Start Learning

In this article, we discussed how buffer overflows are encountered, the varieties of overflows that can materialize, and ways to control the flow of execution to our arbitrary code. We have also covered various forms of prevention mechanisms that can be taken to thwart buffer overrun attacks. Memory management and CPU registers have also been covered, giving us the elementary knowledge indispensable to detect and exploit buffer overflow vulnerability. We looked into actual exploits on how they were written and where the control on the flow of execution had taken place. Understanding all these sections will aid us in the future when it comes to analyses, debugging, and exploiting the buffer overflow vulnerability.