Applied Reverse Engineering with IDA Pro

This editorial is committed to subverting the essential security restriction mechanisms of a native binary executable by employing the IDA Pro Dissembler. This paper is basically elaborating a very complex mechanism of reverse engineering among the previously demonstrated papers, yet because it is a very exhaustive and long process, it claims epic proportions of patience and proficiencies in Machine code instructions -- but it is very interesting and challenging. We have mostly focused on .NET reverse engineering so far, which is relatively an easy task, instead of native binary reversing, because the source code is straightforwardly manipulated by dissemblers. This article would surely assist the aspirants who are repeatedly seeking a step by step tutorial on IDA Pro reverse engineering, because no such paper is perfectly crafted in laborious fashion so far.

[download]

This article showcases the particulars of these contents:

Live Binary Sample Target

Target Analysis with IDA Pro

Cracking the Target

Alternative way of Tracing

Final Note

There are 'n' numbers of approaches for reverse engineering, and picking the appropriate one depends on the target program, the platform on which it runs and on which it was developed, and what kind of information you're seeking to extract. Generally speaking, there is one fundamental reversing methodology: offline analysis, which is all about taking a binary executable and using a disassembler to convert the machine code into a human-readable form. Reversing is then performed by manually reading and analyzing parts of that output. Offline code analysis is a powerful approach because it provides a good outline of the program and makes it easy to search for specific functions that are of interest.

The disassembler is one of the most significant reverse engineering apparatuses. Essentially, a disassembler decodes binary machine code into a readable assembly language code. The disassembler merely decodes each instruction and creates a textual representation for the code. It is trivial to say, the specific instruction encoding format and the resulting textual representation are entirely platform-specific. Each platform provides a different set of instructions and registers. Therefore a disassembler is also platform-specific (even though there are a couple of disassemblers that contain specific support for multiple platforms).

Essential

• The target file
• IDA Pro Interactive Dissembler
• PE signature verifier
• Assembly Language skills

Live Binary Sample Target

This time, I have chosen a target binary which is being applied over reverse engineering, and its origin is in fact totally unknown to us. This executable basically first validates the user identity by asking the password. If user enters the correct information, then he would be able to proceed; otherwise it echoes the wrong password message over the screen. But we have obtained this binary from some unauthentic sources. Thus we are not provided with the user password token key which would probably have part of a license key. Luckily, we have only the executable, not the source code, so that we can figure out the mechanism implemented behind the scenes.

Figure 1.1

So the only possibility left to carry on without buying the license code is to reverse engineer this binary file using IDA Pro dissembler. It is indispensable to confirm either this target binary is a standard Windows PE file or belongs to some other platforms, because it is mandatory for a binary file to have the PE file signature; otherwise, IDA Pro won't dissemble it. So to achieve this, we can assist PE Explorer to obtain the signature information of this binary as follows in the file Header section.

Figure 1.2

The aforesaid figure clearly expresses that this binary has a Windows PE signature and hence could be dissembled by IDA Pro, which we are going to elaborate in the next section.

Target Analysis with IDA Pro

Well, we have obtained the origin information of the target file so far using PE explorer. As we have stated earlier, reversing with IDA Pro is truly a laborious task, because we have to encounter trivial machine code. We don't have the source code, rather only the binary executable. However, we first decompile or dissemble the binary using IDA Pro in order to comprehend what mechanics are implemented implicitly. Thus, launch the IDA Pro software, and it will ask to choose the prototype of a new disassemble file:

Figure 1.3

After configuring the disassemble prototype as a new project, the IDA prompts to open the target binary as in the following figure.

Figure 1.4

Right after choosing the target file, IDA Pro displays a screen dialog which stated three type of a file to be reversing as PE File, DoS Executable File, and Binary file. These file types basically point out the platform on which they were developed. In our scenario, the PE file is best fit-in because we have chosen a Windows 32 console application as per the figure 1.2; Apart from that, we can organize other option such as processor type, kernel and DLL renaming as follows:

Figure 1.5

Finally, click OK. Don't bother ahead because IDA Pro prompts a couple of alert messages, open in two different dialog windows. On the whole, much internal processing is done before opening a target file.

As you can grasp the RED BOX in the following figure 1.6, we have a function window which enumerates all the methods and routines used in this executable. The graph window creates the control execution in flow chart format stated in the RED BOX. It provides a drag-able and moveable dashed rectangle box which can let us reach anywhere in the code execution.

In the BLUE BOX, it shows the decompile code in assembly code format and most importantly, we can access any segment of code such as entry point, containing text string, binary pattern and marked position just by dragging the pointer in the first RED BOX.

Figure 1.6

The important point to note here is that the Debugger menu would only be visible if the target file has the correct PE signature; otherwise it remains invisible. That's why is better to identify first the signature of the target file using PE explore. We shall accomplish the task of logic tracing by debugging the decompiled file. We however choose the appropriate debugger. In our case, we pick Local Win32 debugger as follows:

Figure 1.7

After ensuring the mandatory configuration, it is time to correlate with the actual mechanism by using the arrow mentioned in the following figure 1.8; it basically requires some hit and trail to find out the actual execution code path by dragging the pointer. The alert string message mentioned in figure 1.1 assist us to figure out the execution path. In fact, this target file shows numerous execution paths, and some of them are useful in the context of reversing, and the remaining ones are useless. So, the code flow that shows the alert message "Enter the password" is very significant to the reversing point of view, because this is the key value or entry point by which we can trace the essential code.

Figure 1.8

After moving the pointer to a specific location, we can find the actual mechanism logic flow as following. It typically shows the control flow when we enter the wrong password value.

Figure 1.9

The logic path flow mentioned in the aforesaid figure usually does not fit in the work area window. For this purpose, we can move the dashed rectangle in the graph overview by dragging it to reach a specific segment as follows:

Figure 1.10

After moving the pointer to the appropriate location, we have found the decompiled code in assembly language format. Here, we can easily assume that this program prompts the user to enter the password by the scanf method mentioned in the RED BOX. Then this value is compared to a predefined string value (which is password) using the strcmp method. The test eax register is holding the value 0 or 1 which would come based on the string comparison. Finally, the jnz instructs the compiler to directly jump to the false segment branching, which is location 411444.

Figure 1.11

If the eax register contains the value 0, then the condition would be true and the code execution directed to the box highlighted by cyan color. If it has a value of 1, then the control flow diverts toward the false condition block as follows:

Figure 1.12

If the user enters the correct password, the following assembly code segment would be executed, in which first the "congratulations" message would be displayed along with the actual password information as follows:

Figure 1.13

As we have stated earlier, the code flow instructions are huge in quantity, so we have to move the dashed rectangle from time to time in order to reach the specific code block. This time we move to a false condition block as follows:

Figure 1.14

If the eax register doesn't contain the value 0, then the execution is diverted toward the following figure block where the "Wrong Password! Try again" message will be print on the screen as shown below.

Figure 1.15

Finally, no matter what value the eax register holds, the compiler always executes the following assembly instruction, where the getch method is encountered every time as follows:

Figure 1.16

So, we have successfully disassembled the target assembly code to correlate to the actual mechanism running behind the scenes. We have come to a conclusion that the eax register value is the key hack. If its value is 0, then we entered into the true condition code block; otherwise we entered into the false condition block.

Cracking (Reversing) the Target

So, the eax register value would be the key interest for the reverser to subvert the password mechanism. If we change that value manually during debugging, then we can reach the true condition block even if we enter the wrong password information.

In this encounter, our leading objective is to collapse the jump statement (jnz) execution so that the calling of method loc_411444 won't happen. To do so, run this application in debugging mode. However, place a breackpoint at eax instruction by using F2. The instruction would be submerged in red box as follows:

Figure 1.17

Then, run this executable by Start Process (F9) from Debugger. Again, a couple of windows appeared, then disappeared as usual.

Figure 1.18

After that, the target starts to execute in DoS mode, because it is a console application. Here, it asks the user to enter the password as per its functionality, but unfortunately we don't have the password. So, just enter any value as a password and press enter.

Figure 1.19

The moment we press enter after entering the bogus password value as test, the execution is transferred to the IDA View-A, and we reach the instruction where we placed the breakpoint earlier. Then we have to move ahead manually by pressing the step into (F7) and we step forward to the jump instruction. Here, we notice that the green arrow (in the RED BOX) started blinking, which points out that the execution is about to transfer to the B31444 method block. If we didn't do anything, the wrong password message display as usual.

Figure 1.20

The execution is transferred to B31444 (false condition) block because eax register has a value as 1. The transmission to the false condition block happens due to the ZF value as 0. If this value would have 1, then the true condition block would execute.

Figure 1.21

However, if we modify the value of ZF to 1, then we could control the code path execution to a true condition block. In order to change this value, right click over value and select Modify value as:

Figure 1.22

Here, just replace the ZF register value from 1 to 0 and click OK.

Figure 1.23

The moment the ZF register value is changed to 1, we notice that the execution flow is instantly changed to the true condition block by repeatedly blinking the red arrow in the BOX as follows:

Figure 1.24

Now, go to the Debugger menu and press the Run until return option, which ends the execution after reaching the end of the code. Bingo!!! The target binary is showing the congratulation message even if entering the wrong password value. In addition to that, this program shows the original password value as ajay.

Figure 1.25

So, we successfully subvert the password security mechanism just by diverting the binary code flow execution. But to get rid of a common misconception, we have to reverse engineer this target file, which is actually not patched. The password restriction is still in place, because we have not modified the corresponding bytes so far.

Alternate way of Tracing

Identification of the program entry point is always complex in IDA Pro because it shows raw assembly code. We have applied such a process by moving the arrow as in Figure 1.8. But this is a very cumbersome task. There is another way which eases the task of identifying such entry points.

A program displays some string to carry on the execution ahead or to assist the user to control the execution. Just consider the following figure: It asks the user to enter the password by prompting the "Please enter the password" string.

Figure 1.26

This string could be beneficial in terms of finding the entry points. So go to Text search and enter this string in the box, which finds such entries in the assembly code as follows:

Figure 1.27

Alternatively, the string window displays all the in-built strings value of the target program. Just double click over the "Please enter the password" string, which redirects us to its assembly code.

Figure 1.28

Here, we didn't find the code execution in flow chart format, but in actual assembly code. So we reached the location where the string is located. Place the breakpoint here by F2 as earlier. Therefore, follow the operation or steps as performed earlier from Figure 1.18:

Figure 1.29

Final Note

This rare piece of information illustrated the process of disassembling as well as reverse engineering tactics over a native binary by using IDA Pro disassembler. We have seen the importance of the register values of binary code to correlate with actual program implementation and what role they can play in the reversing process. We have subverted the password mechanism just by modifying the value of the ZF register, which is connected to the eax register. The tutorial that applies to the binary target is dedicated to reversing the logic flows only, not patching the byte code associated with mechanism. There is no permanent change in memory related bytes; if we run this target again without IDA Pro, the password mechanism is still there, and it is not bypassed yet. We will discuss byte patching in detail in the next article.