Reverse Engineering Tutorial 1

7/26/2009

by Stephen Dunn

Part 1: Tools of the Trade

Reversers use many tools to accomplish a given task. Fortunately, most are free or have free counterparts. I will provide you with links here, but some programs I will leave to you on your own to obtain because they cost money. Here are brief descriptions of what we will be using:

OllyDebug v1.10 Final - http://www.ollydbg.de/odbg110.zip
This is the RE standard debugger. Olly v2 is out, but I'm sticking with 1.10 final for now. You will need this to trace your way through live executing code line by line, do memory patching, and all sorts of other things. Probably your most useful resource. Yes I am aware that there is a newer version in beta testing, but I highly recommend the older one for now because you will need many of the plugins designed for it. The plugins can be found on the Olly page, here are a few you should download and install: OllyDump, Asm2Clipboard, Bookmarks, Command Line (should come installed), FindCrypt, AttachAnyway, HideDebugger.

IDA Pro Freeware v4.9 - http://85.17.92.154/files/idafree49.exe
Often the second line of attack when regular Olly analysis fails, a IDA is a professional dissassembler that I find very useful. It will help you immensely and really requires an entire tutorial on just its use alone. You will learn this program as you go. It is what takes the EXE and shows you the raw assembly, analyzes function calls, and lets you see how a program is laid out internally.

PEiD v0.95 - http://www.brothersoft.com/....zip
PEiD allows you to easily identify programs used to pack up the binary (UPX, 7Zip, etc.). It also has many useful plugins available and I find it very useful. There are alternatives if for some reason you don't like it (just like for everything else here), Google is your friend.

Sysinternals Suite - http://download.sysinternals.com/Files/SysinternalsSuite.zip
Lots of useful live monitoring tools, including ProcMon, RegMon, FileMon, and TcpView. The makers of the Sysinternals programs have been bought out by Microsoft, so don't be surprised if you see them hosting the files.

HIEW v7.45 - ?
Well, here's one of my favourite programs. Probably the simplest, most straightforward patcher available, it's a must have. There are also alternatives available here, and many other versions of HIEW. Yes, again, I know HIEW v8.00 is out. But who wants to spend money? Not to say that any version is free, but some keys are easier to find than others.... You'll have to use your brain here. Find a copy. Even the makers of HIEW know the game, see their website here. Hey, if you've got the cash buy a copy, it's one of the most useful programs ever written.

WinRAR v3.91 - ?
Again, find this on your own. In my opinion, the simplest, fastest, most versatile zipper/unzipper around. The official website is here.WinRAR is a powerful archiver and it's native format is the .rar file.

Hex Workshop v6 - ?
You'll have to find this on your own to, or any good hex editor will do. I happen to really like Hex Workshop and its context menu integration.

Part 2: Your First Reverse

I highly recommend you scan all of your downloads, even from trusted sources. You should compare checksums for further confidence. Also note that the source code for the tutorial is included (created using VS 2008 SP 1). (I plan on expanding on this tutorial later for a keygen tutorial, so that is why there is an empty function checkCode().)

Ok, so first things first. Run the program (Tutorial1.exe)! We are hit with a nag screen...

Nag

That will be task one, removing the nag screen. The important info here is the caption and text which we may be able to locate in the file later. Don't worry about taking notes and being diligent yet, this first run through is just to get an idea of what we want to do.

Next we reach a screen with two edit controls, a name and a registration code box. Here's a recommendation from experience: Pick a name and a unique code (not your name and not 1234) which you will remember and always use these. Why? Because it will make your life easier when your tracing through code in Olly looking for that one give-away to help you locate the key routine. If you use different info everytime it might not catch your eye or you simply might not remember what you entered after ten minutes of pressing F8. Here I use 40Hex and Tutorial1 as my name and code respectively, and receive the following error:

Invalid

There is some very important information here. The error message caption and text is crucial to finding the routine we want. Many modern applications do not even display an error message upon failure for this very reason, but here we're lucky :). We now have enough information to go on. Hit quit and pop open PEiD, click the ... button, and select Tutorial1.exe. Let's see what information we can glean from good ol' PEiD:

PEiD

Ahh! It's packed... That means we can't just open it in Olly and start the debugging. First we need to unpack it so the data doesn't appear garbled. We can see that it's been packed with UPX, probably the most popular and powerful packer around. There are two ways of solving this problem, the easy way and the hard way. Both will work in this case, but we'll take the easy for now. You'll need to download UPX (might as well do it now, you'll need it again anyway), which you can get here: http://upx.sourceforge.net/download/upx303w.zip. Drop upx.exe into the same folder as tutorial1.exe and open a console. Navigate the console to the appropriate folder (e.g. "cd \reversing\myfirsttut"), and type: "upx -d tutorial1.exe". You should see:

UPX

The -d tells upx to decompress the file. This should be successful, and the program should be noticably larger by a few kb. You may be wondering what the "hard way" was. Well, it is possible to manually unpack an executable by stepping through with the debugger (Olly), dumping the unpacked version and correcting the file offsets. I'll save that for another time. Now that you have your nice unpacked file, scan with PEiD again. You should see "Nothing found *" which means that no packers were detected. Note that this does not mean that there are no packers, just none that PEiD can find. Our next step will be to open up OllyDebug and load the program. In Olly click File->Open and select the tutorial1.exe file. Also note that it us very wise to keep multiple copies of the executable you're working on, as one can always accidentally become corrupted. You may also need to restore from the original from time to time, or test different patched out. Anyway, you should now be looking at something like this:

Olly

If you're here, so far so good. Now it's time to find that nag. Here is some basic Olly knowledge: F8 - Step Over, F7 - Step Into, Ctrl + F9 - Run until return. If you aren't sure what that means don't worry, I'm coming to that. What exactly is all that you're looking at up there? What do JMP, RET, MOV, CALL, POP, and PUSH mean? What are EAX, EBP, ECX, EDX, ESP, etc? This is where assembly knowledge comes in. If you know assembly, fantastic. If you don't: Don't sweat it, you learn the basics quickly. Besides, you don't have to be fluent in it to reverse. Just the basic concepts will be necessary. Here I'm going to list what each code does in layman's terms:

MOV X, Y Moves data from location Y into location X.
JMP X Program flow will jump to location X and continue from there.
CMP X, Y Compare values X and Y. This is usually followed by a jump or test instruction.
JE X / JNE X JE and JNE mean "jump if equal" and "jump if not equal" respectively. Usually after a CMP.
JZ X / JNZ X JZ and JNZ are functionally the same as JE and JNE (they stand for jump if zero)
RET Returns program flow to previous location just after the CALL command.
CALL X Stores current location and calls the function at X location. Returns back when RET is reached.
PUSH X Pushes the data at location X onto the stack.
POP X Pops the data from the top of the stack into location X.

Ok, but what about EAX, ECX, etc? What are those? Those are called registers. They are small memory locations located in the processor that keep track of the important information about your program, like where program execution is currently located in memory, where the top of the stack is, loop counters, pointers, etc etc. I'm not going to explain each one here because then I'll just be writing an assembly tutorial. If you want an assembly tutorial, the definitive guide known as the Art of Assembly can be downloaded here: http://webster.cs.ucr.edu/AoA/Windows/aoapdf.zip. I wouldn't recommend reading the entire thing unless you really want to. Just use it as a reference when you don't know something, otherwise you'll never make it through the other side of those PDFs alive.

Back to where we were, OllyDebug. If you're in Olly, hold down F8 to cruise through the beginning of the program. You will quickly notice yourself inside of a loop because program flow will be jumping from the bottom of the page back to the top and vice versa. At the bottom of Olly you should see a string which represents the program's physical location on your hard drive getting smaller and smaller by one character each loop. However, one part of that loop right in the middle stares out at me very clearly, and should to you to. It's this part:

Entry

Why does this stand out? Because we can clearly see two alternate paths for our program. On the one hand, we have a function simply titled (by Olly) tutorial.00401130, and the other is exit. Exit is the Win32 API exit() which terminates the program. We both know that the program isn't going to terminate here, but this point is critical because it tells us that a check will occur that could exit the program. Armed with that knowledge, set a breakpoint at 00401382. How? There are multiple ways. In the Command Line box, you could type: "bp 00401382", at which point you should see that address turn red. You can also double-click on the address 00401382 in the left hand column of addresses, or right click on the line: "CALL tutorial.00401130" and click Breakpoint -> Toggle. You can also highlight the line by clicking once and then hitting F2. All different ways of accomplishing the same thing, I normally prefer double-clicking when I can. Now that our key line is red, click the play button at the top menu bar of Olly so that program flow will run until it hits that line and stop. You could alternatively press F9 to run the program. You should now have the program stopped right where you want it. Press F7 to step into the critical code. You should be at this point:

Nag Code

Nice! We've found the nag, but how do we get rid of it? If we walk through with F8 we will see the message, but that's no good. What do we do? Well young reverser, time to perform your first reverse! Right-click on our target line 00401133 and click the Assemble option. We click here because that first PUSH ESI begins the process of calling MessageBoxA, the Win32 API for displaying message boxes. We need to get past all of the those pushes and the call to MessageBoxA. You should now see this:

Assemble1

Now in the box type "JMP 00401148", because that will tell the program to jump past the message to line 1148, which if you look below is the line just after the message.The dump in Olly should now look like this:

NoMoreNags

Excellent! We've managed to disable our nag message. If you press F9 to run the program, it now starts up with no annoying message. Now the bad news: There's a catch. If you exit Olly and run the program the nag comes back! How do we get our patch to save? That will come later. For now you must understand that changes you make to a program in Olly are only in memory, not on the hard drive. That is why Olly is so important, we can see our program executing live. What do we do now? Write down that memory address 00401142 along with the instruction NOP so that you know what to do to it later in HIEW. Now it's time to move on dear readers, the excitement has passed and it's time to tackle bigger things. It's reg code time. Here's where you're going to learn some more valuable skills: Setting breakpoints on code that you can't easily locate. A Windows program that has a window always has what's called a callback function. This function is called every time a message is sent to the window. These messages contain information about an action, say if the user clicks in the window, or moves it, or resizes it, or closes it. There are tons of possible messages, but we just want to find the code that handles the registration code check. How do we do that? We set a breakpoint. Now here is where some programming knowledge really comes in handy because we have to set breakpoints on APIs until we guess correctly. Here's a list of common dialog API calls:

I usually have success with one of those when trying to track down the dialog API I want. How do we set the breakpoint? By using the Command Line plugin in Olly. Type in the box "bp GetDlgItemTextA" and hit enter. You have set a breakpoint on that API every time it's called. From now on you will immediately be brought right into the API call when it is called. Here we have another problem: How do we find our way out of the API call and into the code we want? By using Ctrl+F9 to run until a return. Here's a summary of what you want you should have done so far and what you need to do now: Disable the MessageBox by nopping out the code. Set a breakpoint on GetDlgItemTextA (you could have picked any from the list, I'm just saving you some time because it's the most common and the one I used). Run the program and break on the API. Once you set the breakpoint and press F9 to run the program you should then enter in a username and password into the appropriate boxes. Then click Register. Olly breaks into the API. Now press Ctrl+F9 to execute until we hit the RET assembly instruction which will take us to the end of the API call and then press F8 to get back out into the real code.You will see the following on your screen in Olly:

BP on API

What you see in front of you is a classic reverse waiting to be accomplished: On the one hand is a failure message, the other contains success. There are once again multiple ways to accomplish the final part of your task. A note about this tutorial: The "registration code" in this tutorial is hard-coded. That means that only one code will actually work, as opposed to a typical modern application which generates a key based on a users name. I realize that this tutorial is unrealistic in this sense, but it is for learning purposes only and we will handle keygens in a different tutorial. The method we will use here to modify the code is still accurate and would work on an ordinary keygen type app anyways. What we are going to do is trick the program into allowing ALL keys entered to be the correct one. A side note: You can clearly see just after the call to GetDlgItemTextA the message "YouAreNowL33T". If you enter this into th reg box and hit register you will receive the following message:

Success!

Congratulations! You have found the real code. Most real programs will not simply contain the registration code in plaintext like this however, so I guess you were just lucky this time. But instead of just quitting here, let's instead learn how to perform another valuable reversing technique: Changing the code to always go where we want. How? Start pressing F8 (slowly) until you reach the following line:

FailCode

If you look at the bottom where I have circled you can see that the jump is NOT taken. Is that bad or good? To find out, look up to the other part I circled and take a look at the following line: "JE SHORT tutorial.004010FF". What does JE mean? You will notice the "TEST EAX, EAX" code just before the JE. This code is checking the return value of a Strcmp() function called previously to compare the values of two strings, the real code and the one entered. Since we both know that the code you entered is not the correct one, the fact that the jump is NOT taken must be a bad thing. How can we test our theory to change the jump and hopefully "correct" the program? Right-click the JE line and click the Assemble option. Change the JE at the beginning to JMP and click assemble. This tells the program to always jump no matter what is entered. Press F9 to run the program and we are greeted with the sweet taste of success! Well done if you've made it this far! You have just figured out how to completely "crack" this program. So how do we make our changed permanent? You must first write down the code we need to change:

Ok, so now we know what must be done. Close out Olly. Open up HIEW. Use the keyboard arrows to scroll to the unpacked exe and press enter to open it up in HIEW (it is easiest if you just place your tutorial1.exe file into the same folder as HIEW.exe). You should now see this:

HIEW

What we want to see is the assembly code so we can modify it. Press F4, then press the down arrow to select Decode mode. Press enter. You can now see the assembly. We now need to navigate to the two locations we need to modify. Press F5 for GOTO mode, then type (please note the leasing period, it IS necessary): ".00401133", press enter. You should now see the line PUSH ESI. Press F3 to enter edit mode, and now look carefully at all of the addresses. You should notice that they have all changed because you are now looking at relative addresses, so take note of the new relative offset of the line just after the call. It is line 548. Then F2 to enter assembly code editing mode. Type "JMP 548", press enter to apply the change, then ESC to get back out. Now press F9 to make the changes to the program permanent.

In edit mode:

EditMode

After patched:


Now press F5 and type ".004010EF" and enter to get to the next patch location. Repeat the steps above to change the JE at 004010EF to a JMP. You don't need to change any addresses here. Press F9 to save and F10 to quit. Now run your cracked program! No nags, and all serials return success.

Well I hoped you liked this tutorial and learned something. Shoot me an email at dunn.stephen{at}gmail.com if you have anything to say. Criticism is welcome. Now get cracking!

Part 3 - Resources

A great crackme website to practice and learn: http://www.crackmes.de

A fantastic RE website with tools and resources: http://www.reteam.org/

I'll add more here later, send me your good links too.