Malware analysis

Reverse engineering packed malware

In this article, you’ll get a better understanding of what a packed executable is and how to analyze and unpack malware. Finally, you’ll get to know the top packers used in malware.

Become a certified reverse engineer!

Get live, hands-on malware analysis training from anywhere, and become a Certified Reverse Engineering Analyst.

Start Learning

What are packed executables?

It’s an executable that has been compressed firstly to minimize its file size, but often to complicate the reversing process. Not to be confused with standard compressions (rar/zip). 

Packed executables are standalone files that can be executed while still compressed. A packer uses standard compression techniques (LZO, LZMA, …) on the file; of course, the OS won’t recognize these code modifications, but the packer appends an unpacking routine to the executable. When it is run, the unpacking routine unpacks the code and loads it into memory in its original state.

Figure 1: Generic example of packed executable

 

Analyzing packed malware

1. Set up the virtual environment

To analyze a malware in general, you must first isolate that malware in a virtual environment (VMware or VirtualBox) with the analyzes tools, in order not to infect your main machine. For more details, check out the following links

• OALabs Malware Analysis Virtual Machine, OALabs
• How to Get and Set Up a Free Windows VM for Malware Analysis, Zeltser Security Corp.
• Malware Analysis: First Steps — Creating your lab, Medium

2. Analysis tools

Next, you need to have your analysis tools set up. In case you're not sure, here’s a list:

Scanners/packer identifier

• DiE
• PEiD

Disassemblers/debuggers

• OllyDbg
• IDA
• Cutter

PE tools

• ImpREC
• PEBear

Other

• Process Hacker (Monitor system resources)
• Wireshark (Network protocol analyzer)
• HxD (Hex Editor)
• Resource Hacker (Extract resources from executables)
• VirusTotal (Online analysis of malware samples and URLs)

Once you're done, create a snapshot of the current VM's state.

3. Analysis

Now that everything is set up, you can start analyzing the packed sample. First, you analyze the malware in VirusTotal to check if it’s recognized by antiviruses:

Figure 2: Detection of a sample in VirusTotal

 

As you can see, the file has been recognized by the majority of antivirus systems.

Now analyze the sample using DiE (Figure 3) to check if it’s packed and the packer it uses:

Figure 3: Scan (DiE)

 

Figure 4: Entropy (DiE)

 

As you can see, there are three sections (Figure 4), which proves it’s packed by UPX. Even if it checks as not packed, that’s because malware authors sometimes use custom packing or modified versions of packers or scramble the code (change section names, add junk code and so on).

Manual unpacking

In general, unpacking is as follows:

1. Trace the code

That’s not difficult if you encounter a simple packer like UPX or ASPack. But when it comes to advanced packers, it can be hard, as they can use all kinds of tricks to prevent you from unpacking.

2. Find the original entry point (OEP)

There are multiple ways of getting the OEP. This depends on the packer and its decryption algorithm, or whether you’re familiar with entry points generated by different compilers. Or you can identify where the unpacked code is being written during tracing. You can also use common APIs such as GetCommandLine or GetVersionEx to get closer to the OEP.

In the case of UPX, the decryption routine starts as follows (Figure 5) and ends with Figure 6:

Figure 5: UPX decryption routine start (Cutter)

 

Figure 6: End of UPX decryption routine (using Cutter)

 

That last jump (jmp section.UPX0) will lead straight to the unpacked code.

Try loading the sample in OllyDbg (Figure 7), set a breakpoint on that jump and run the app, the app will break, press F7 and you’ll find yourself in the original unpacked code.

Figure 7 - Original Unpacked Code (OllyDbg)

 

3. Dump the unpacked code

After finding the original OEP, you’ll have to dump the memory image of binary.

Using OllyDbg, there a plugin called OllyDump (Figure 8). Click “Dump debugged process.”

Figure 8: OllyDbg’s OllyDump plugin

 

A box shows up (Figure 9). Uncheck Rebuild Import and click Dump.

Figure 9 - OllyDump dump box

 

Notes: 

1. Note the OEP = 1000
2. Keep the process running in OllyDbg

4. Rebuilt import

Now you have to rebuild imports using an ImpRec (Figure 10), launch the tool and select the running process (malware.exe).

Figure 10: ImpREC

 

Update the field OEP to 1000 and click IAT AutoSearch. You’ll get a message saying the following:

Figure 11: ImpREC IAT AutoSearch

 

Click Ok and then click Get Import. You should get the Imported functions with their status (valid:YES) (Figure 12):

Figure 12: ImpREC imported functions

 

5. Save unpacked file

Now that you’ve got all imports resolved, all you have to do is click “Fix Dump in ImpRec” and choose the file you saved before, and you’re done. 

There are other variations not covered in the above steps that would require more work. For example, some packers don’t fully resolve imports. There is also the Stolen Code technique, which makes it harder to find and recover OEP, and plenty of tricks that packers use to make reversing harder. Still, the mentioned steps can cover most of what’s out there. 

Top 5 popular packers used in malware

This section shows the top packers used by malware. The analysis was done by ShadowServer using the SigBuster (Figure 13) tool.

Figure 13: SigBuster inspection

 

1. Custom/unknown

Malware authors use custom packers most of the time. For example, Allaple Polymorphic Packer is not really a packer but a standalone malware program, but its polymorphic engine uses techniques similar to packing.

2. UPX

An open-source executable packer supporting a number of file formats from different operating systems. It’s used a lot by malware authors because it’s open-source and can be modified.

3. NsPack

A wonderful tool which can compress 32bits and 64bits exe, dll, ocx, scr files. Also supports C#.

4. ASPack

An advanced Win32 executable packer. It makes Windows executables smaller by up to 70% (compression ratio is higher than ZIP standard by 10-20%). It also uses self-modifying code, which makes software breakpoints useless.

5. Install systems

This is mostly used by ransomware and FakeAVs to hide their code and avoid detection; for example, NSIS or Inno Setup.

Conclusion

This was just a simple example of a packed malware that uses nothing but compression. In the real world, there is more sophisticated malware that really know how to hide its code, so anyone doing reverse engineering must learn to keep up with this danger.

Sources

1. Malware Forensics Field Guide for Windows Systems, Digital Forensics Field Guides (Cameron H. Malin, Eoghan Casey, James M. Aquilina)
2. Revealing Packed Malware, Secure Systems
3. Awesome Malware Analysis, Ryan Shipp (GitHu