General security

Cracking 64bit Binaries

May 18, 2015 by SecRat

Keygenning is a process of finding a valid key for a program. It is used for cracking/piracy. Most of the cracking has been documented on x86, there haven’t been many articles on x64 cracking.

In this article, we will show you how to keygen a Linux x64 bit application on a Linux computer.

For purpose we will use

1: Linux machine ( 64bit mint box)
2: EDB debugger
3: IDA Disassembler
4: Compiler to write a key generator

5: Fill out the form below for the files associated with this article

Let’s run file command to check the type of file.

file r5

r5: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.24, BuildID[sha1]=86bf854ce620288567d153883d4609163485d34d, not stripped

From the output, we see the build version, and it is a dynamically linked file.

~/Desktop $ nm r5
0000000000601109 B __bss_start
00000000006010e0 D buf
000000000040069d T check_password
0000000000601109 b completed.6972
0000000000601060 D __data_start
0000000000601060 W data_start
00000000006010a0 D delta
00000000004005e0 t deregister_tm_clones
0000000000400650 t __do_global_dtors_aux
0000000000600e18 t __do_global_dtors_aux_fini_array_entry
0000000000601068 D __dso_handle
0000000000600e28 d _DYNAMIC
0000000000601109 D _edata
0000000000601110 B _end
0000000000400894 T _fini
0000000000400670 t frame_dummy
0000000000600e10 t __frame_dummy_init_array_entry
0000000000400a80 r __FRAME_END__
0000000000601000 d _GLOBAL_OFFSET_TABLE_
w __gmon_start__
0000000000400500 T _init
0000000000600e18 t __init_array_end
0000000000600e10 t __init_array_start
00000000004008a0 R _IO_stdin_used
w _ITM_deregisterTMCloneTable
w _ITM_registerTMCloneTable
0000000000600e20 d __JCR_END__
0000000000600e20 d __JCR_LIST__
w _Jv_RegisterClasses
0000000000400890 T __libc_csu_fini
0000000000400820 T __libc_csu_init
U __libc_start_main@@GLIBC_2.2.5
00000000004007b6 T main
0000000000601080 D master
U printf@@GLIBC_2.2.5
U puts@@GLIBC_2.2.5
U random@@GLIBC_2.2.5
0000000000400610 t register_tm_clones
00000000004005b0 T _start
U strcmp@@GLIBC_2.2.5
U strcpy@@GLIBC_2.2.5
U strlen@@GLIBC_2.2.5
0000000000601110 D __TMC_END__

x64 assembly basics

x64 consists of extended register set and some extra instructions are added as well.

Following is the list of added registers in x64

r8, r9 , r10, r11, r12, r13, r14, r15

Lower 32 bits of r8 can be accessed by r8d, lower 16 bits can be accessed by r8w and lower 8 bits can be accessed by rb8

and more over RIP (instruction pointer) can be directly accessed.

All the register in x64 are 64bit in sizes . RIP is also 64bit but Current implementations only support 48 bit linear addresses.

In addition to normal registers it also added SSE registers namely from xmm8 – xmm15

If any data movement operation is performed on EAX, it zero extends the higher 32 bits of RAX register.

Some added instructions are lodsq, stosq etc.

For the purpose of debugging, we will use an x64 debugger known as EDB on Linux. This debugger is similar to ollydbg (windows) and is quite easy to use .Following is the default pane of EDB

Argument passing in x64 is quite different from x86 itself

Arguments are passed in registers RDI, RSI, RDX, RCX, r8 and r9 rest of the parameters are passed on the stack

Navigation is simple just like ollydbg

Running our crackme file just like that gives us the following output

~/Desktop $ ./r5
Usage: ./r5 password

Maybe plaintext isn’t good after all.

Which gives us a hint that it requires a password, which we have to figure out

Opening it in a disassembler gives us an idea of what is happening around. Apparently it is looking for a parameter and is passing it to a function

Clearly you can see that it passing argv[1] as a parameter to function check_password()

The first hint is about the length of the input string, which should be equal to the length of “this_is_not_even_interesting_its_garbage”

.data:00000000006010E0 ; char buf[]
.data:00000000006010E0 buf db ‘this_is_not_even_interesting_its_garbage’,0
.data:00000000006010E0 ; DATA XREF: check_password+1C#o
.data:00000000006010E0 ; check_password+3C#o …
.data:00000000006010E0 _data ends
.bss:0000000000601109 ; ===========================================================================

and is checked here

call _strlen ; Call Procedure
mov rbx, rax
mov edi, offset buf ; “this_is_not_even_interesting_its_garbag”…
call _strlen ; Call Procedure
cmp rbx, rax ; Compare Two Operands
jz short Go ; Jump if Zero (ZF=1)

After that, this string is replaced by our own input string

mov rax, [rbp+passcode]
mov rsi, rax ; src
mov edi, offset buf ; "this_is_not_even_interesting_its_garbag"…
call _strcpy ; Call Procedure
mov [rbp+VarCheck], 1
jmp loc_400791 ; Jump

After this operation program goes in a loop and loop body is skipped if value at index of variable delta is zero

movzx eax, delta[rax] ;

If not, it performs some mathematical operations on the input strings leveraging on delta and other parameters

which can be represented in C language as

x = (random() % delta[index] ) + 1;
delta[index] = delta[index] – x;
var_check = var_check ^ (unsigned int )delta[index] ;

random() call is not initialized with srand() so it can be predicted easily.

Finally, after the 40 rounds of loop, the mutated string is compared against “this_aint_that_simple_but_good_luck_haha” and if it is equal, “password OK” message is printed

Now to calculate that string we can perform the exact opposite on this string to get out key

We can use the following C program to do so.

#include <stdio.h>

unsigned char delta[] =
3, 253, 3, 249, 0, 3, 6, 0, 241, 0,
250, 7, 22, 235, 8, 252, 246, 2, 254, 243,
4, 19, 1, 234, 237, 15, 253, 240, 242, 15,
12, 243, 241, 12, 7, 0, 5, 14, 10, 4,

unsigned char buff [48] ;
int main(int argc, char **argv)
int index = 0;
int var_check = 1;
unsigned char x = ‘x00’;

strcpy(buff, "this_aint_that_simple_but_good_luck_haha");

while ( var_check )

index = 0;
var_check = 0;
while ( index < 40)

if (delta[index])
x = (random() % delta[index] ) + 1;
delta[index] = delta[index] – x;
var_check = var_check ^ (unsigned int )delta[index] ;

buff[index] = buff[index] + x;


// if zero

printf("%sn", buff);

Compiling and running this program gives us the following output:


~/Desktop $ ./r5 “well_done_now_go_on_irc_and_ask_for_more”

password OK

Posted: May 18, 2015
View Profile

SecRat works at a start-up. He's interested in Windows Driver Programming.