Incorporating Custom Shellcode Into The Metasploit Framework

Carl Benedict
November 29, 2011 by
Carl Benedict

Writing shellcode can be a somewhat time consuming task. Once considered a dark art, shellcoding has become a part of the vocabulary of modern IT personnel. With IT Security becoming more and more important within organizations today, it's becoming practical for many people to not only know what shellcode is, but to also understand what it does, how it does it, and how to craft your own. Modern tools, such as the Metasploit Framework, have reduced the work necessary to write shellcode and to leverage it to exploit vulnerabilities.

This is achieved through code reuse in the form of Metasploit modules and payloads, a powerful set of provided tools, and the ability to extend the console itself.

What should you learn next?

What should you learn next?

From SOC Analyst to Secure Coder to Security Manager — our team of experts has 12 free training plans to help you hit your goals. Get your free copy now.

What should you learn next?

What should you learn next?

From SOC Analyst to Secure Coder to Security Manager — our team of experts has 12 free training plans to help you hit your goals. Get your free copy now.

This article will explore how to utilize and incorporate your own shellcode for use with the Metasploit Framework. We will focus more on the end-to-end process than on any individual piece. I will add tidbits that I have found to be useful or particularly relevant. However, this article will not teach you the finer points of writing shellcode, x86 assembly basics, or the Windows API. These topics have been covered in many articles and books. Instead, this article will give you a higher-level understanding of how the pieces fit together, where many of the challenges lie, and what tools can be used to aide you along the way. In order to focus on the bigger picture, we will utilize simple example code, pre-existing code, and other readily available data.

The Metasploit Framework is known for being a handy tool for penetration testers everywhere.

This article will discuss using the latest Metasploit Framework to assist with leveraging your own shellcode to exploit Windows vulnerabilities. We'll cover the different methods that can be used to generate your shellcode as well as tools to streamline the process. This article will not cover how to discover vulnerabilities in applications. While knowing how to find exploits could be considered a prerequisite to performing the tasks we will cover, that topic is outside of the scope of this article.

Target audience:

Who should read this article? Security researchers, those beginning exploit development, or anyone who would like to learn more about incorporating their own custom shellcode for use within Metasploit may all find this article worth a read.

What You Will Need:

To follow along with the exercises in this article, you will need to acquire the following software. All software can be obtained at no cost.

While I recommend these tools, we will not dive deep into their use here. I recommend HxD as it's a free, powerful, open-source hex editor for the Windows platform. However, as there are no special requirements here, any hex editor will do. If you already have similar tools installed in an existing environment, you should still be able to follow along without much trouble.

At the time this article was written, the latest version of the Metasploit Framework (herein after MSF) is the 4.x series. Version 4 was released on August 1st, 2011. We will be using the open source "Framework" version. This version was chosen because of it's proven ability to be a powerful platform for writing exploits as well as being consistent with older versions of MSF. For more information on the Pro, Express, and Community editions, take a look at the Rapid7 website.

We won't be covering the installation of MSF as that process has already been documented for Windows, Linux, and OSX. We will be using MSF on the Windows platform. I chose this platform because we will be working with shellcode for a Windows application. This facilitates a more fluid development process as development and testing can be performed on the same machine.

Our Vulnerable Sample Application:

To illustrate how to work with shellcode and incorporate it into our own MSF module, we will use a sample application with an intentional vulnerability. This application will accept user input using the 'gets' C function and place it into a buffer on the stack. The length of the input is not checked or truncated to the buffer size, which makes it vulnerable to a stack based buffer overflow.

Below is our sample C program:



int main() {

char buff[4];

printf(“Please enter the last 4 digits of your SSNn”);

printf(“: “);




You can compile this code with the following command:

gcc –o vuln.exe vuln.c

Let's imagine that this is a legacy application used in your organization by the human resources department. The original developers have long since departed the company. Management has decided that this application must run until the systems that support become completely obsolete.

You've discovered the buffer overflow in this application and want to show management how risky it is to continue running this software in your organization. You've spent time crafting some custom shellcode by hand to exploit this vulnerability and would like to incorporate it into the MSF to make things easier for your team while testing. In order to ensure that you have the attention of management, you also plan to create various payloads to exploit this vulnerability.

Understanding the Role of Shellcode:

If you have never written or analyzed shellcode before, you may wonder why special attention is paid to how it is formatted. After all, it's just a stream of opcodes, right? While that is true, exploiting a vulnerability using shellcode has limitations. Depending on the nature of the vulnerability, you may not be able to use certain characters in your shellcode.

As a general rule, you will always have to be concerned about NULL bytes (i.e. x00). This is due to the nature of the C language, which uses the NULL byte as a string terminator. Factors such as input filters, available memory or stack space, and environmental settings may require you to be more creative when crafting your shellcode. As with many other topics, there is often more than one way to accomplish your goal. When crafting your own shellcode, you will have to be mindful of many potential limitations.

A few of the possible road blocks you may face when writing shellcode are memory size limitations, finding your shellcode in memory (this is often referred to as "egg hunting"), and system protection mechanisms. Common system protection mechanisms that you will encounter are Address Space Layout Randomization (ASLR), Data Execution Prevention (DEP), and canary markers. These mechanisms were designed and implemented to increase security in the Windows operating system and to mitigate certain types of exploits. As you can see, writing shellcode is only one piece of this complex puzzle. All of these items must be considered when crafting shellcode. They will influence how you craft your payload.

Using the Metasploit Framework:

The current MSF uses the Ruby framework for modules (payloads, exploits, etc). Modules contain definitions that have meaning within the MSF. Payload data is, not surprisingly, hex-encoded. Each byte is prefixed with a "x" delimiter to indicate that it is a hexadecimal value. Within the MSF, exploit modules incorporate payload modules.

The general format for a Ruby-based MSF module is as follows:

  • Required 'include' module definitions
  • Class definition
  • Function / method definitions

We will focus on the aspects of the exploit and payload modules that affect our shellcode.

The MSF documentation states that user modules should be placed in the 'user_module_directory'. This is, by default, set to '~/.msfX/modules' (where X = major MSF version number). On the Windows platform, this equates to the following:

C:Documents and Settings<user>.msfXmodules

To clarify, <user> is equal to your username and .msfX is equal to .msf3 or .msf4 (in our case, .msf4). It is within this directory that we will re-create the MSF module tree. It is not necessary to replicate the entire tree, however, we should place our module in the same folder within this structure that would make the most sense. For a sample exploit module, the exploitwindowsmisc folder is appropriate. The full path to our module on disk would then be:

C:Documents and Settings<user>.msfXmodulesexploitswindowsmiscsample.rb

This module can be loaded using the following command in the MSF console:

msf> use windows/misc/sample

We will also need a payload module to place our shellcode into. The separation of exploit and payload facilitates modularity and code reuse. This makes using the MSF for exploitation very flexible and powerful. As you will see, the payload modules are not drastically different from exploit modules. Our payload module will be stored and used as follows:

C:Documents and Settings<user>.msfXmodulespayloadswindowssinglespayload.rb

This module can be configured to act as our payload when using the sample exploit module by using the following command in the MSF console:

msf (sample)> set PAYLOAD windows/payload

Shellcode Generation:

The following section briefly covers tools and methods that can be used to assist you with writing shellcode.

Method 1: METASM
While metasm is not new (it has been around since MSF version 3.x), it is still a very powerful way to quickly write and encode shellcode without having to actually code and compile it by hand using traditional binary tools. metasm is also a handy tool since it is part of the MSF and our goal is to use a custom payload within MSF. It's worth noting that at least as of v4.0.0, metasm utilizes the Intel syntax, and not the AT&T syntax. Generating shellcode with metasm is as simple as typing assembler into the console and hitting return. While the metasm console will recognize each language construct on its own, I will be separating each instruction with a semicolon for clarity. As an example, if you type "xor eax,eax; push eax; nop; nop;" into metasm, it will return the string "x31xc0x50x90x90". In this way, metasm acts as an on-the-fly compiler, which gives you output in the format needed for MSF payloads.

Method 2: NASM
The Netwide Assembler (NASM) is another tool you can use to write shellcode. This is a sound approach since our goal is to feed the CPU our crafted string of anarchy. What better tool is there for generating ASM than an assembler? When using nasm to generate shellcode, it will be necessary to create flat/raw output. This is achieved with the following command:

nasm -o vuln_shellcode.asm vuln_shellcode.bin

This will generate a flat binary file that does not incorporate a specific executable format, such as Windows PE. In our case, this is exactly what we want. Our code will be executing within an existing Windows application. It does not require the data needed to be a stand-alone program.

Now that we have compiled our sample shellcode, we can open up the flat binary in our favorite hex editor. From this point, we have a few choices as to how we convert the data for use within MSF. We can copy our hexadecimal string from HxD, paste it into our favorite text editor, and use some search-and-replace-fu to append the "x" delimiters. We could also use xxd for conversion on the command-line. Once you have went through this process a few times, you will realize just how powerful the metasm console is. This is especially true after needing to make only small changes to your shellcode.

Method 3: By Hand
I've listed this final method for crafting shellcode as I feel it is not only most powerful way from a control perspective, but it also provides benefits that the other methods do not. Writing shellcode from scratch by hand each time is generally not practical. The amount of effort required to craft a series of several hundred bytes of code on-the-fly that executes flawlessly can be several orders of magnitude more difficult than using tools to assist you. However, this is also the beauty of using this method. Writing shellcode by hand will quickly make you more intimately familiar with the target platform. You will gain a much better understanding of the structure of code, platform-specific opcodes, endianness, as well as an appreciation for more advanced methods. Security researchers and exploit developers often write from scratch, when needed. Alternatively, you can simply use or modify existing shellcode such as that included in the MSF to fit your needs. This is a practical approach which can save you time and allow you to develop a reusable code base. However, it is imperative that you understand what your base shellcode actually does before you blindly incorporate it into your work. The bottom line: if you really want to gain proficiency with a specific platform, this method is essential.

Method 4: Use MSF
The MSF includes many payload modules that allow you to use the 'generate' command to create shellcode on the fly. Like other MSF modules, you set a few options and you're on your way. Since we are discussing bringing shellcode into the MSF, we will not cover this option in depth. More information about this topic can be found on the Metasploit website.


Once you have converted your shellcode to the appropriate format, incorporating it into the payload module is simple:

Set some basic options in your exploit module:

There are many options that can be set to define a module in the MSF. The options we are most concerned about right now relate to our payload. The 'BadChars' parameter defines which hexadecimal characters we can't use due to input filters or other limitations. In this case, we have defined only x00 (NULL), but you'll likely need to define more in other scenarios.

The 'Space' option sounds a bit misleading at first. It is, in fact, the max size you wish your shellcode to be. If you do not set the 'DisableNops' option to 'false', any encoders that are run will expand your shellcode to this max size by filling it with NOPs (x90).

Incorporate your payload into your payload module:

As you can see, the payload module contains some of the same parameters as the exploit module. For our purposes, formatting and incorporating our shellcode string into the payload section is sufficient.

The sample shellcode listed above is formatted nicely. It is easy to read and is split into 16 character strings. While this is not necessary, it can be much easier to deal with than a long string that runs hundreds of characters to the right on one line. In the end, the coding style is up to you.

Once you have completed writing or using a template for your exploit module and filled in your payload module, you can then:

msf (sample)> exploit

Become a Certified Ethical Hacker, guaranteed!

Become a Certified Ethical Hacker, guaranteed!

Get training from anywhere to earn your Certified Ethical Hacker (CEH) Certification — backed with an Exam Pass Guarantee.

Become a Certified Ethical Hacker, guaranteed!

Become a Certified Ethical Hacker, guaranteed!

Get training from anywhere to earn your Certified Ethical Hacker (CEH) Certification — backed with an Exam Pass Guarantee.

In Closing

Whether you're porting an exploit into the MSF or creating modules for a new proof-of-concept, I hope this article provided you with a place to start. For a more in-depth look at what the MSF can do for you, take a look at the Developer Guide.

Carl Benedict
Carl Benedict

Carl Benedict is a Security Engineer and consultant with a passion for vulnerability research and development, web application security, and reverse engineering. He currently works for an organization listed as one of the "200 Best Small Companies" by Forbes. As well as working independently, Carl has worked for small businesses, Fortune 500 companies, and the Federal Government.

He is a Certified Ethical Hacker and researcher at InfoSec Institute. When he is not engaged in IT security issues, Carl enjoys exploring all that Northern California has to offer. You can follow him on Twitter @infinitenigma.