Virtualization Security

Virtual Machine Introspection in Malware Analysis – Use Case

March 26, 2019 by Youness Zougar

To determine the behavior of a piece of malware, we will develop a script (based on LibVMI functions) that will allow us to trace the Kernel APIs executed by a malware and their arguments.

1. Preparation

After choosing the domain name of the machine, create the file containing the dictionary and determine the malware file name to be scanned.

The following command is launched from the hypervisor:

./monitor_api w6164-1 /tmp/w6164-1.json malware.exe

This script will take several entries as argument:

  • w6164-1: The domain name of the virtual machine under Xen
  • /tmp/w6164-1.json: The file containing the functions/structures and the corresponding offsets
  • malware.exe: The malware we want to analyze

Below is the code allowing the initialization of our system of introspection:

// Create altp2m view that will altered with breakpoints
xc_altp2m_set_domain_state(xch, domain_id, 1);
xc_altp2m_create_view(xch, domain_id, 0, &shadow_view);

2. Breakpoints Insertion

We first define the APIs we want to monitor. Take, for example, the following three APIs:

  • NtCreateFile: Allows file creation/opening
  • NtSetValueKey: Create or replace the value of a registry key
  • NtDelayExecution: Delays execution (this API is executed when the Sleep API call is made from user mode). It can be used as an evasion technique

For more information on these APIs, you can refer to the MSDN documentation.

Afterwards, we insert the breakpoints in the altp2m view for each API.

// Add breakpoints on the monitored APIs in the altp2m view
xc_altp2m_set_domain_state(xch, domain_id, 1); uint8_t trap = 0xcc;
vmi_write_8_pa(vmi, (shadow << 12) + shadow_offset, &trap);

3. Callbacks Initialization

In this example, we will rely on three types of callbacks that we will initialize.

3.1. Interrupt Events Callback

This callback is triggered each time a breakpoint is trapped. It will retrieve the desired information through the function process_event (). This function will allow to identify the name of the API that reached the breakpoint and its arguments.

// Register interrupt event callback, which will be executed each time a breakpoint is trapped
SETUP_INTERRUPT_EVENT (& trap_event, 0, interrupt_event_cb);
vmi_register_event (vmi, & trap_event);
event_response_t interrupt_event_cb (vmi_instance_t vmi, vmi_event_t * event)
    process_event (...);
    event-> slat_id = 0;
   event-> interrupt_event.reinject = 0;

3.2. Step-by-Step Events Callback

This callback is triggered directly after the interrupt callback is executed. This allows us to execute, step-by-step (as with debuggers), the instruction of the breakpoint from the unaltered view. Then we resume the execution from the altered view. In the case where several vCPUs are allocated to the virtual machine, it is necessary to loop on the number of vCPUs and create a callback for each.

// Register singlestep event callback which will execute one instruction
int vcpus = vmi_get_num_vcpus (vmi);
for (i = 0; i <vcpus; i ++)
    SETUP_SINGLESTEP_EVENT (& singlestep_event [i], 1u << i, singlestep_event_cb, 0);
    vmi_register_event (vmi, & singlestep_event [i]);

event_response_t singlestep_event_cb (vmi_instance_t vmi, vmi_event_t * event)
    event-> slat_id = shadow_view;

3.3. Memory Read/Write Events Callback

This callback is triggered when a read or write is made on one of the memory pages of the shadow copy. If this is the case, a change of view is instantly made on the unaltered view. This avoids a BSOD when the system or malware performs an integrity check.

// Register memory event callback, which will change the view from altp2m to the unaltered view
// In order allow other actions (e.g., never alter an integrity check, otherwise it will BSOD)
SETUP_MEM_EVENT (& mem_event, ~ 0ULL, VMI_MEMACCESS_RW, memory_event_cb, 1);
vmi_register_event (vmi, & mem_event);

event_response_t memory_event_cb (vmi_instance_t vmi, vmi_event_t * event)
event-> slat_id = 0;

For more information on the functioning of these callbacks, check

4. Retrieving Information

When a breakpoint is reached, it is confronted with the offset and API dictionary. The desired information is then retrieved (arguments from the API) and execution is continued.

Let’s take as an example NtCreateFile. To get the arguments of this function, we first define its prototype:

NTSTATUS NtCreateFile (
IN ACCESS_MASK DesiredAccess,
IN ULONG FileAttributes,
IN ULONG ShareAccess,
IN ULONG CreateDisposition,
IN ULONG CreateOptions,

The name of the accessed file and the access rights are among others the information that may interest us concerning this API. The file name is in the OBJECT_ATTRIBUTES structure. In order to parse this structure, we will use the dictionary previously generated, which contains the offsets of the various entries of the structure:

"Attributes": [24, ["unsigned long", {}]],
"Length": [0, ["unsigned long", {}]],
"ObjectName": [16, ["Pointer", {"target": "_UNICODE_STRING"}]]

We will do the same for the other structures.

There is no need to modify the arguments of the API, even if it is possible to do so. The goal is to let the malware run without modifying its behavior. We retrieve the interesting information on the fly and then, at the end of its execution, we analyze the information collected and determine its behavior.

Below is an events log generated during malware analysis. Only information referring to potential malicious behavior is kept.

{"target":"REGISTRYUSERS-1-5-21-336141597-709016518-532797093-1001SOFTWAREMICROSOFTWINDOWSCURRENTVERSIONRUNPersistence","process" : "C:UsersFrancisDesktopsample.exe","pid":2616,"value":"C:UsersFrancisAppDataRoamingWindowsconhost.exe","api":"NtSetValueKey"}, 

5. Conclusion

Aside from using introspection in virtualization, security solutions developers took advantage of this technology to use it in other areas such as malware analysis. Its features are essential assets for dealing with advanced and sophisticated forms of malware. The use of introspection in malware analysis is not yet fully democratized, since the technology requires quite advanced skills on Windows and Linux internals.

A special thanks to Tamas (aka @tklengyel), who is doing a great job out there with the LibVMI framework.


Posted: March 26, 2019
Articles Author
Youness Zougar
View Profile

Youness Zougar is an IT Security Expert, especially in Reverse Engineering and Malware Analysis. He is also a Windows Kernel Drivers developer and co-founder at SAFE-Cyberdefense. In order to offer companies the best level of security, he developed with his teammates an Endpoint solution called SAFE Endpoint Security based on a defense in depth technology applied to workstations.