VMWare ESX Audit & Analysis, Part 1

November 8, 2012 by Blackhat Academy

The VMware ESX source code (from 2004 according to VMware, Inc.) was partially leaked on November 4, 2012. Following due-diligence to determine the impact, the source code has been analyzed and audited for a number of common vulnerabilities. The source code was seemingly either only partially obtained or only partially disclosed by the attacker. It is difficult to determine which, however it is important to note that none of the files within the leaked archive ended abruptly, indicating that it is in fact possible that the attacker may have a much more complete copy. Due to the age of the code released, one can infer that either one of VMWare’s archive servers could be recently compromised, or that the code may have been released by a disgruntled employee.

It is our belief that by examining the existing potential for vulnerablity within the newly leaked code base and cross-referencing it with the current version of ESX, we can determine the true impact of the breach. In this installment (Part 1), we’ll focus on how VmWare works, common programming mistakes in the released version, and a bit of our auditing methodology. Its important to keep in mind that because the leak was only partial, there is no current way to fully rely on the leaked code to identify vulnerabilities. Functions and data types remain undefined in certain areas, and entire headers are missing in others. In our next installment, we’ll test the potential bugs identified here against the latest version of ESX.

When running the preliminary audit, the first step was to grep for known-to-be-vulnerable or similar functions throughout the project. We were able to narrow it down from 300,000 lines of code (the entirety of the leak) to a few thousand lines or “highlights” where the vulnerabilities are most likely to be. Checking for usage of strcpy and other vulnerable functions is a good traditional first-step when encountering any unfamiliar code. The full results will be available in the appendix, however the simple statistical information is available here:

String handling functions

41 strcpy
3 strcat
189 strcmp

Format string functions

20 printf
19 fprintf
0 sprintf

Memory handling functions

31 malloc
7 calloc
397 memset
222 memcpy

The code was analyzed for a variety of common vulnerabilities including, but not limited to, stack and heap overflows, double free bugs, and string or file truncation amongst other problems. The source had a multitude of bugs which may or may not have had a security impact since then. Several potential security vulnerabilities were discovered; however, due to incomplete source code they cannot be verified until the next article. As such, remember that this article focuses on identifying programming and code habits and patterns – familiarizing ourselves with VMWare’s development team traits, and potential vulnerabilities.

None of the potentially vulnerable code in this document is confirmed as vulnerable. While all of the vulnerabilities remain unconfirmed, we can still discern patterns and habits of the developers who worked for VMWare at the time of this copy of ESX’s development. A single developer is usually prone to making the same mistakes. Where there is a single bug of any type, there are usually more of them wherever the same developer was involved.

The following commands were used to generate the appendix files:

find -type f -name *.c -exec grep -iHn bstrcpy( ‘{}’ ; -o -name *.h -exec grep -iHn bstrcpy( ‘{}’ ; &> esx_audit/strcpy.txt

find -type f -name *.c -exec grep -iHn bstrcmp( ‘{}’ ; -o -name *.h -exec grep -iHn bstrcmp( ‘{}’ ; &> esx_audit/strcmp.txt

find -type f -name *.c -exec grep -iHn bstrcat( ‘{}’ ; -o -name *.h -exec grep -iHn bstrcat( ‘{}’ ; &> esx_audit/strcat.txt

find -type f -name *.c -exec grep -iHn bprintf( ‘{}’ ; -o -name *.h -exec grep -iHn bprintf( ‘{}’ ; &> esx_audit/printf.txt

find -type f -name *.c -exec grep -iHn bfprintf( ‘{}’ ; -o -name *.h -exec grep -iHn bfprintf( ‘{}’ ; &> esx_audit/fprintf.txt

find -type f -name *.c -exec grep -iHn bsprintf( ‘{}’ ; -o -name *.h -exec grep -iHn bsprintf( ‘{}’ ; &> esx_audit/sprintf.txt

find -type f -name *.c -exec grep -iHn bmalloc( ‘{}’ ; -o -name *.h -exec grep -iHn bmalloc( ‘{}’ ; &> esx_audit/malloc.txt

find -type f -name *.c -exec grep -iHn bcalloc( ‘{}’ ; -o -name *.h -exec grep -iHn bcalloc( ‘{}’ ; &> esx_audit/calloc.txt

find -type f -name *.c -exec grep -iHn bmemset( ‘{}’ ; -o -name *.h -exec grep -iHn bmemset( ‘{}’ ; &> esx_audit/memset.txt

find -type f -name *.c -exec grep -iHn bmemcpy( ‘{}’ ; -o -name *.h -exec grep -iHn bmemcpy( ‘{}’ ; &> esx_audit/memcpy.txt

And right off the bat, there are several potential vulnerabilities. Before jumping in, it’s important to note that many times functions that would normally be safe, if properly checked, are in fact not properly checked. A very common mistake is to confuse the order in which a memory or string copy occurs – leading a developer to mistakenly provide the wrong length argument to the function. There were several instances of memcpy() that fit this description, and it was not likely alone in its backwards implementation and size checking:

Accidental Mis-sizing
./net/vmxnet2_vmkdev.c Line 721 memcpy(&frp.outputFilter.ladrf, &dd->LADRF, sizeof(dd->LADRF));
./hardware/smp.c Line 489 memcpy(oldApicIds, apicIDs, sizeof(apicIDs));
./hardware/pci.c Line 531 memcpy(newDevices[func], dev, sizeof(*dev));
./main/dump.c Line 419 memcpy(&dumperMsgReply, hdr, sizeof(*hdr));
./main/dump.c Line 1389 memcpy(writeBuffer, &info, sizeof(info));
./main/dump.c Line 1422 memcpy(writeBuffer, &info, sizeof(info));
./main/mod_loader.c Line 627 memcpy(&(list->desc[numModules]), &desc, sizeof(desc));
./main/mod_loader.c Line 627 memcpy(&(list->desc[numModules]), &desc, sizeof(desc));
./main/rpc.c Line 2046 memcpy(cnxList, &pendingCnx, sizeof(pendingCnx));
./main/world.c Line 1914 memcpy(world->kernelGDT, &defaultGDT, sizeof(defaultGDT));

memcpy() is not the only C function confused this way by the VMWare developers in 2004.
Inspecting snprintf, strncat, strncpy, and other functions is a very good idea.

These backwards length checks were identified by hand, because they were quite obvious. Many more occurances of this problem likely exist within the code without referencing another argument, specifically in the specified length, and therefore will be harder to identify than those shown above. These would be found primarily through manual code inspection, as opposed to automated methods. Typically, while preliminary reviews can be performed in an automated fashion, there are many more in-depth possibilities when placing code under the human-controlled magnifying glass.

The best nugget of information that VMWare’s source code has freely provided was a bit of commentary explaining the way that virtual machines start and stop. It explained that when a virtual machine is booted its “world” is loaded; the world contains a group and a “UserCartel” structure. The world becomes the group leader and its ID is the same as the virtual machine’s ID. As more virtual machines are booted, they will share the ‘UserCartel’ and ‘World’/’Group’ leader. When a virtual machine monitor powers on in a virtual machine, the monitor worlds are created and then they join the same world group of the virtual machine via a linked list. The first monitor created becomes the monitor leader. The monitor leader will not be cleaned up until all other monitors have exited.

The risk associated with the leak of the source code is somewhat obvious. While the intellectual property hasn’t much suffered due to the date on the released source, it will be interesting to determine its effect on VMWare’s next move and reputation in the virtualization industry as Red Hat Linux moves forward with OpenStack. The leak’s true technical impact is contingent on the status of today’s version of ESX and how it has changed since 2004. If VMWare has changed its implementation as a whole at this point, the impact is minimal. If the code base has changed very little, the poor practices VMWare used in 2004 speak for themselves regarding the impact and the inherent vulnerability presented by the leak to the product. Next time we will check to see if any of the bugs discovered are listed in the CVE records between 2004 and today, and identify those before testing anything against the latest version. After that, we’ll find out the impact of the leak for sure, running tests for the potential vulnerabilities discovered in this segment against the latest available version of VMWare ESX Server.

Posted: November 8, 2012
Blackhat Academy
View Profile

Blackhat Academy is an organization dedicated to educating the public on matters regarding secure programming, networking, administration, internet development, and generally what it means to be a hacker and survive in today's world. It is not a group focused on hacking: rather, it is just a group of hackers that want people to learn. A common misconception is that a hacker is a person that commits computer crimes. Blackhat Academy does not commit any crimes or even teach people how to commit crimes. What one learns from them should be used appropriately towards defending oneself. The bad guys already know this stuff; Blackhat Academy (like InfoSec Institute) exists to enlighten the rest of us.