Reverse engineering

Assembly programming with Visual Studio.NET

MASM is maintained by Microsoft and is an x86 assembler that consumes Windows and Intel syntax to produce a COFF executable. It is compatible for both 16 bit and 32 bit sources. Fortunately, Microsoft's Visual Studio IDE endorses MASM programming tasks just by making a couple of project property changes. The prime objective behind this article is to introduce the power of assembly code in terms of speed and full control over programs which are typically not seen in other programming languages. Even though there are numerous editors and software available to do such a task in a standalone way, the aspirant system or security programmers who are only limited to .NET software IDE so far can enter into the real system programming world by using none other than visual studio IDE.

Prerequisite

Become a certified reverse engineer!

Get live, hands-on malware analysis training from anywhere, and become a Certified Reverse Engineering Analyst.

Start Learning

In this article, we would get an understanding about creating both EXE and DLL using MASM with Visual Studio. So, the newbies should to have a brief knowledge of these technologies:

• Visual Studio 2010 or Later Version
• MASM (Microsoft Macro Assembler) SDK Library
• Basic Assembly Coding Competency
• VC++

[download]

Developing EXE using MASM

We shall demonstrate assembly programming by creating a simple Windows executable which typically shows "Hello World!" in a message box the moment it is initiated. It is very tricky to do such an implementation because Visual Studio 2010 IDE doesn't offer any explicit templates for writing assembly code like C#, VC++ and VB.NET programming languages. It in fact has an in-built option to compile or run assembly programs.

Opening new project

We shall have to create a VC++ project solution which later is accompanied with an assembly code file. Hence, open Visual Studio and choose an Empty Project of VC++ template type. There is no need to create a sub-directory for this empty solution, so uncheck the corresponding check box as follows:

Once the test_masm of VC++ type solution is created, go to the solution explorer and right click to choose Build Customization option as follows:

The Build Customization options open up the MASM compiler options which uncheck by default. This is the key option which must be enabled in order to edit and compile the native assembly code file.

Assembly coding

As we have stated earlier, VS 2o1o doesn't provide assembly file templates, however choose a project from the solution explorer and right click to add a text file which will be provided a *.ASM extension as follows:

Now, a blank text.asm file is added to our test_masm solution. Open it and paste the following assembly code, which is responsible for displaying a message box, as follows:

[c]

.386 ; Tells MASM to use Intel 80386 instruction set.

.model flat,stdcall ; Flat memory model

option casemap:none ; Treat labels as case-sensitive

include masm32includewindows.inc

include masm32includekernel32.inc

includelib masm32libkernel32.lib

include masm32includeuser32.inc

includelib masm32libuser32.lib

.data ; Begin initialized data segment

MsgBoxCaption db "Win32 Assembly Programming",0

MsgBoxText db "Hello World!!!Welcome to ASM Programming under CLR",0

.code ; Beginning of code

start: ; Entry point of the code

invoke MessageBox, NULL, addr MsgBoxText, addr MsgBoxCaption, MB_OK

invoke ExitProcess, NULL

end start

[/c]

The assembly code file is written, but keep patience, this is not ready to compile or execute because some of important project settings are still remaining.

 

Mandatory project configurations

Successful execution of an assembly code file with Visual Studio IDE depends on an external library file, which will be available from MASM SDK. Hence, choose project Properties by right clicking it from the solution explorer. Here, choose General by expanding Linker and in the Additional Library Directories, insert the path of include, lib and macros directories as follows:

Next, come to the Input section in the Linker and mention the reference of masm32.lib file as additional dependencies:

It is not required to generate a manifest file for such manipulation, hence disable it as follows:

Now, come to System from the Linker and set Windows in the subsystem section as follows:

Finally configure the code entry point as the start from the Advanced option in the Linker, which determines the code execution flow. We can identify the entry point of the ASM file from the .code section.

Now come to the Microsoft Macro Assembly section from the solution properties which appears the moment when we add an assembly file in solution directory, otherwise it shall be hidden. Here, set the directory name where the MASM SDK was installed earlier as follows:

Finally, everything is ready and the solution is compiled. If the whole configuration is correct, then a test_masm.exe file is created in the Debug folder of the solution.

Testing and Debugging

It is time to test the executable. When the exe is clicked, a "Hello World!" message box would appear as follows:

We can even debug the assembly code by inserting a breaking point as a specific location, and through the Register window in the Debug menu, we can observe all the CPU registers with corresponding flags as follows:

We shall cover the advanced debugging of an application in later articles. The following image shows the assembly code in debug mode which helps us to understand what is happening behind the scenes.

Although this section is not relevant to this article, but just for knowledge point view, we can disassemble any C++ file to its corresponding ASM code. The Visual Studio IDE is inbuilt with a Disassembly option, which is very helpful to detect a run time bug such as buffer overflow in the code via converting the source code file to an assembly code file as follows:

Developing DLL using MASM

In the previous section, we have seen how to create an EXE file using MASM with VS 2o10. We can also develop a library (DLL) by using MASM programming much like other technologies such as C#, VB, and C++. Therefore, the method can be utilized in the other client application in that created DLL. The procedure of generating a DLL is almost the same as EXE but requires some subtle configuration. First of we have to set Configuration type as DLL in the General section because now we are dealing with DLL. Such modification can happen from solution properties as:

And as we all know, DLL files are libraries which contain method definitions. Entry point is typically absent in the DLL file. Hence we have to change this setting as follows:

Finally, add a text file as masmlib with ASM extension in the solution like earlier and place the following code, which typically contains a testing method which will show some alert during the load and unload of DLL in the client program as follows:

[plain]

include masm32includemasm32rt.inc

.data

dLoading BYTE "HELLO ,DLL is Loading.....", 0

dUnloading BYTE "DLL is un-loading.???????", 0

dOrdinal BYTE "Good-Bye", 0

.data?

hInst DWORD ?

.code

testingMethod proc hInstDLL:DWORD, fdwReason:DWORD, lpvReserved:DWORD

.if fdwReason == DLL_PROCESS_ATTACH

invoke MessageBox, HWND_DESKTOP, offset dLoading, NULL, MB_OK

push hInstDLL

pop hInst

mov eax, TRUE

ret

.elseif fdwReason == DLL_PROCESS_DETACH

invoke MessageBox, HWND_DESKTOP, offset dUnloading, NULL, MB_OK

.elseif fdwReason == DLL_THREAD_ATTACH

.elseif fdwReason == DLL_THREAD_DETACH

.endif

ret

testingMethod endp

ProcByOrdinal proc

invoke MessageBox, NULL, offset dOrdinal, NULL, NULL

ret

ProcByOrdinal endp

end testingMethod

[/plain]

Finally, compile this program and test_masm_dll. The DLL file would be created in the Debug folder which can referenced in the C++ program or in the MASM client program itself.

Become a certified reverse engineer!

Get live, hands-on malware analysis training from anywhere, and become a Certified Reverse Engineering Analyst.

Start Learning

Conclusion

So, we have seen how to create both EXE and DLL files using MASM programming languages employed with visual studio IDE. In fact, such a task could be achieved by hard-core MASM SDK but .NET programmers typically fear assembly programming due to strange syntax and platforms. Assembly language programming opens a new horizon of advance coding in terms of faster code executing, exploit writing and shell-coding. Programmers are often comfortable with Visual Studio due to having numerous in-built features and functionality. Hence, this article is dedicated to those professionals who are planning to shift towards system programming without leaving the .NET framework.