Breaking Bits
Search…
Vulnerability Discovery
CTF
Firmware Emulator
Exploit Development
Modern Vulnerability Research Techniques on Embedded Systems
This guide takes a look at vetting an embedded system (An ASUS RT-AC51U) using AFL, angr, a cross compiler, and some binary instrumentation without access to the physical device. We'll go from static firmware to thousands of executions per second of fuzzing on emulated code. (Sorry no 0days in this post)
Asus is kind enough to provide the firmware for their devices online. Their firmware is generally a root file system packed into a single file using squashfs. As shown below, binwalk can run through this file system and identify the filesystem for us.
1
$ binwalk RT-AC51U_3.0.0.4_380_8457-g43a391a.trx
2
​
3
DECIMAL HEXADECIMAL DESCRIPTION
4
--------------------------------------------------------------------------------
5
64 0x40 LZMA compressed data, properties: 0x6E, dictionary size: 8388608 bytes, uncompressed size: 3551984 bytes
6
1174784 0x11ED00 Squashfs filesystem, little endian, version 4.0, compression:xz, size: 13158586 bytes, 1492 inodes, blocksize: 131072 bytes, created: 2019-01-09 11:06:39
7
​
8
​
Copied!
Binwalk supports carving the filesystem out of the firmware image through the
-Mre flags and will put the resulting root file system into a folder titled squash-fs1
$ ls
2
40 _40.extracted squashfs-root
3
$ ls squashfs-root/
4
asus_jffs cifs2 etc_ro lib opt rom sys usr
5
bin dev home mmc proc root sysroot var
6
cifs1 etc jffs mnt ra_SKU sbin tmp www
7
​
Copied!
Motivation
The LD_PRELOAD trick is a method of hooking symbols in a given binary to call your symbol, which the loader and placed before the reference to the original symbol. This can be used to hook function, like
malloc and free in the case of libraries like libdheap, to call your own code and perform logging or other intrumentation based analysis. The general format requires compiling a small stub of c code and then running your binary like this:1
LD_PRELOAD=/Path/To/My/Library.so ./Run_Binary_As_Normal
Copied!
I wanted to try a trick I saw online to create a fast and effective fuzzer for network protocol fuzzing. This github gist shows a PoC of creating an LD_PRELOAD'd library that intercepts libc's call to main and replaces it with our own.
1
#define _GNU_SOURCE
2
#include <stdio.h>
3
#include <dlfcn.h>
4
​
5
/* Trampoline for the real main() */
6
static int (*main_orig)(int, char **, char **);
7
/* Our fake main() that gets called by __libc_start_main() */
8
int main_hook(int argc, char **argv, char **envp)
9
{
10
// Do my stuff
11
}
12
​
13
/*
14
* Wrapper for __libc_start_main() that replaces the real main
15
* function with our hooked version.
16
*/
17
int __libc_start_main(int (*main)(int, char **, char **), int argc, char **argv,
18
int (*init)(int, char **, char **),
19
void (*fini)(void),
20
void (*rtld_fini)(void),
21
void *stack_end)
22
{
23
/* Save the real main function address */
24
main_orig = main;
25
​
26
/* Find the real __libc_start_main()... */
27
typeof(&__libc_start_main) orig = dlsym(RTLD_NEXT, "__libc_start_main");
28
​
29
/* ... and call it with our custom main function */
30
return orig(main_hook, argc, argv, init, fini, rtld_fini, stack_end);
31
}
Copied!
My thought was to then call a function inside of the now loaded binary starting from main. Any following calls or symbol look ups from the directly called function should resolve correctly because the main binary is loaded into memory!
Defining a function prototype and then calling a function seemed to work. I can pull a function address out of a binary and jump to it with arbitrary arguments and the compiler abi will place to arguments into the runtime correctly to call the function. :
1
/* Our fake main() that gets called by __libc_start_main() */
2
int main_hook(int argc, char **argv, char **envp)
3
{
4
char user_buf[512] = {"\x00"};
5
read(0, user_buf, 512);
6
int (*do_thing_ptr)() = 0x401f30;
7
int ret_val = (*do_thing_ptr)(user_buf, 0, 0);
8
​
9
printf("Ret val %d\n",ret_val);
10
return 0;
11
}
12
​
Copied!
This process is very manual and slow... Let's speed it up!
Setting up
The extracted firmware executables are all mips little endian based and are interpreted through uClibc.
1
$ file bin/busybox
2
bin/busybox: ELF 32-bit LSB executable, MIPS, MIPS32 version 1 (SYSV), dynamically linked, interpreter /lib/ld-, stripped
3
$ ls lib/
4
ld-uClibc.so.0 libdl.so.0 libnsl.so.0 libws.so
5
libcrypt.so.0 libgcc_s.so.1 libpthread.so.0 modules
6
libc.so.0 libiw.so.29 librt.so.0
7
libdisk.so libm.so.0 libstdc++.so.6
Copied!
​DockCross does not support uClibc cross compiling yet so I needed to build my own cross compilers. Using buildroot I created a uClibc cross compiler for my Ubuntu 18.04 machine. To save time in the future I've posted this toolchain and a couple others online here. This toolchain enables quick cross compiling of our LD_PRELOADed libraries.
The target is the asusdiscovery service. There has already been a CVE for it and it proves to be hard to fuzz manually. The discovery service periodically sends packets out across the network, scanning for other ASUS routers. When another ASUS router sees this discover packet, it responds with it's information and the discovery service parses it.
These response-based network services can be hard to fuzz through traditional network fuzzing tools like BooFuzz. So we're going to find where it parses the response and fuzz that logic directly with our new-found LD_PRELOAD tricks.
Pulling symbol information from this binary yields a quick tell to which function does the parsing
ParseASUSDiscoveryPackage:1
$ readelf -s usr/sbin/asusdiscovery
2
​
3
Symbol table '.dynsym' contains 85 entries:
4
Num: Value Size Type Bind Vis Ndx Name
5
0: 00000000 0 NOTYPE LOCAL DEFAULT UND
6
1: 0040128c 236 FUNC GLOBAL DEFAULT 10 safe_fread
7
2: 00414020 0 NOTYPE GLOBAL DEFAULT 18 _fdata
8
3: 00000001 0 SECTION GLOBAL DEFAULT ABS _DYNAMIC_LINKING
9
4: 0041c050 0 NOTYPE GLOBAL DEFAULT ABS _gp
10
..............SNIP....................
11
33: 004141b0 4 OBJECT GLOBAL DEFAULT 22 a_bEndApp
12
34: 00402cec 328 FUNC GLOBAL DEFAULT 10 ParseASUSDiscoveryPackage
13
35: 00403860 0 FUNC GLOBAL DEFAULT UND sprintf
14
...............SNIP.....................
15
​
Copied!
With this symbol in mind we can open the binary up in Ghidra and have the decompiler give us a rough idea of how it's working:
1
undefined4 ParseASUSDiscoveryPackage(int iParm1)
2
{
3
ssize_t sVar1;
4
socklen_t local_228;
5
undefined4 local_224;
6
undefined4 local_220;
7
undefined4 local_21c;
8
undefined4 local_218;
9
undefined auStack532 [516];
10
11
myAsusDiscoveryDebugPrint("----------ParseASUSDiscoveryPackage Start----------");
12
if (a_bEndApp != 0) {
13
myAsusDiscoveryDebugPrint("a_bEndApp = true");
14
return 0;
15
}
16
local_228 = 0x10;
17
memset(auStack532,0,0x200);
18
sVar1 = recvfrom(iParm1,auStack532,0x200,0,(sockaddr *)&local_224,&local_228);
19
if (0 < sVar1) {
20
PROCESS_UNPACK_GET_INFO(auStack532,local_224,local_220,local_21c,local_218);
21
return 1;
22
}
23
myAsusDiscoveryDebugPrint("recvfrom function failed");
24
return 0;
25
}
Copied!
The function appears to be instantiating a 512 byte buffer and reading from a given network file descriptor through the recvfrom function. A quick visit to
recvfrom's manpage reveals that the second argument going into recvfrom will contain the network input, the input we can control.1
RECV(2) Linux Programmer's Manual RECV(2)
2
​
3
NAME
4
recv, recvfrom, recvmsg - receive a message from a socket
5
​
6
SYNOPSIS
7
#include <sys/types.h>
8
#include <sys/socket.h>
9
​
10
ssize_t recv(int sockfd, void *buf, size_t len, int flags);
11
​
12
ssize_t recvfrom(int sockfd, void *buf, size_t len, int flags,
13
struct sockaddr *src_addr, socklen_t *addrlen);
Copied!
This user input is immediately passed to the
PROCESS_UNPACK_GET_INFO function. This function in responsible for parsing the user input and relaying that information to the router. Opening the function in ghidra reveals a large parsing function. This looks perfect for fuzzing!
aa
The next step is interacting with the function and providing input into that first argument. The first step towards running this as an independent function is recovering the function prototype. Ghidra shows the defined function prototype as below.
1
void PROCESS_UNPACK_GET_INFO(char *pcParm1,undefined4 uParm2,in_addr iParm3)
Copied!
Using stub-builder you can take this information
​
Instrumenting asusdiscover
Similarly to the PoC of the LD_PRELOAD main hook shown above, I needed to hook the main function. For uClibc that function is
__uClibc_main. Using the same trick as above, we'll define a function prototype for the function we want to call, then hook uClibc's main function and then jump directly to the function we want to call with our arguments.To make this process easier, I created a tool to identify function prototypes and slot them into templated c code. The current iteration of
stub-builder will accept a file and a given function to instrument. The tool is imperfect and will use radare2 to identify (often wrongly) function prototypes and place them into the c stub.1
$ stub_builder -h
2
usage: stub_builder [-h] --File FILE {hardcode,recover} ...
3
​
4
positional arguments:
5
{hardcode,recover} Hardcode or automatically use prototypes and addresses
6
hardcode Use absolute offsets and prototypes
7
recover Use radare2 to recover function address and prototype
8
​
9
optional arguments:
10
-h, --help show this help message and exit
11
--File FILE, -F FILE ELF executable to create stub from
12
​
Copied!
An example for the command can be seen below. The stub builder uses radare2 for it's function recovery and fails to identify the first argument as a
char* so we need to fixup the main_hook.c.1
$ stub_builder -F usr/sbin/asusdiscovery recover name PROCESS_UNPACK_GET_INFO
2
[+] Modify main_hook.c to call instrumented function
3
[+] Compile with "gcc main_hook.c -o main_hook.so -fPIC -shared -ldl"
4
[+] Hook with: LD_PRELOAD=./main_hook.so ./usr/sbin/asusdiscovery
5
[+] Created main_hook.c
6
​
Copied!
Hardcoded values can be inserted instead. The below command supplies the address, argument prototype and the expected return type:
1
$ stub_builder -F usr/sbin/asusdiscovery hardcode 0x00401f30 "(char *, int, int)" "int"
Copied!
1
#define _GNU_SOURCE
2
#include <stdio.h>
3
#include <dlfcn.h>
4
​
5
//gcc main_hook.c -o main_hook.so -fPIC -shared -ldl
6
​
7
/* Trampoline for the real main() */
8
static int (*main_orig)(int, char **, char **);
9
​
10
/* Our fake main() that gets called by __libc_start_main() */
11
int main_hook(int argc, char **argv, char **envp)
12
{
13
​
14
//<arg declarations here>
15
char user_buf[512] = {"\x00"};
16
//scanf("%512s", user_buf);
17
read(0, user_buf, 512);
18
int (*do_thing_ptr)(char *, int, int) = 0x401f30;
19
int ret_val = (*do_thing_ptr)(user_buf, 0, 0);
20
​
21
printf("Ret val %d\n",ret_val);
22
​
23
return 0;
24
}
25
​
26
//uClibc_main
27
/*
28
* Wrapper for __libc_start_main() that replaces the real main
29
* function with our hooked version.
30
*/
31
int __uClibc_main(
32
int (*main)(int, char **, char **),
33
int argc,
34
char **argv,
35
int (*init)(int, char **, char **),
36
void (*fini)(void),
37
void (*rtld_fini)(void),
38
void *stack_end)
39
{
40
/* Save the real main function address */
41
main_orig = main;
42
​
43
/* Find the real __libc_start_main()... */
44
typeof(&__uClibc_main) orig = dlsym(RTLD_NEXT, "__uClibc_main");
45
​
46
/* ... and call it with our custom main function */
47
return orig(main_hook, argc, argv, init, fini, rtld_fini, stack_end);
48
}
49
​
Copied!
The code above will accept input from STDIN and pass it into the parsing function directly. This enable us to test and get return values of the functions without any networking compoonents required.
Running the code
Cross compiling the shared object using the provided cross compilers is shown below. The resulting file will be named
main_hook.so1
t$ /opt/cross-compile/mipsel-linux-uclibc/bin/mipsel-buildroot-linux-uclibc-gcc main_hook.c -o main_hook.so -fPIC -shared -ldl
Copied!
Using this library is shown below and with my toolchain it doesn't link the libdl library and will result in the error below:
1
$ qemu-mipsel -L /home/caffix/firmware/asus/RT-AC51U/ext_fw/squashfs-root -E LD_PRELOAD=/main_hook.so ./usr/sbin/asusdiscovery
2
./usr/sbin/asusdiscovery: can't resolve symbol 'dlsym'
Copied!
Adding the libdl library to the LD_PRELOAD fixes this problem and resolves the dlsym function.
1
$ qemu-mipsel -L /home/caffix/firmware/asus/RT-AC51U/ext_fw/squashfs-root -E LD_PRELOAD=/lib/libdl.so.0:/main_hook.so ./usr/sbin/asusdiscovery
2
abcd
3
Ret val 4
Copied!
We now have the binary running and it's accepting our input and passing it directly to the function. The next stage is generating a set of valid input data to seed our fuzzer with.
Generating valid input for a test corpus
Sending in random strings of "A"s will not yield new discovered paths through the parsing function. Looking at the function decompilation we can see there is a quick check performed in a funciton titled
UnpackGetInfo_NEW . This is the first function we need to look at, to determine if there are any early exits from initial parses.1
memset(&local_320,0,0xf8);
2
memset(&uStack1000,0,200);
3
iVar28 = UnpackGetInfo_NEW(pcParm1,&local_320,&uStack1000);
4
iVar39 = a_GetRouterCount;
Copied!
This function first checks for a set of magic bytes before continueing. It's looking for "\x0c\x16\x00\x1f" to be the first bytes in network input. Without these magic bytes it will exit early and indicate through it's return code to discard the input.
1
int UnpackGetInfo_NEW(char *user_input,undefined4 *param_2,undefined4 *param_3)
2
​
3
{
4
undefined4 uVar1;
5
undefined4 uVar2;
6
undefined4 uVar3;
7
undefined4 *puVar4;
8
undefined4 *puVar5;
9
undefined4 *puVar6;
10
11
if (((*user_input != '\f') || (user_input[1] != 0x16)) || (*(short *)(user_input + 2) != 0x1f)) {
12
return 1;
13
}
Copied!
Supplying this magic value immediatly returns a different result when running the binary:
1
$ python2 -c 'print "\x0c\x16\x1f\x00" + "A"*100' | qemu-mipsel -L . -E LD_PRELOAD=/lib/libdl.so.0:/main_hook.so ./usr/sbin/asusdiscovery
2
Ret val 1
Copied!
The function returns more than just a single return value based on the parse or unpack. There appears to be checks on lines 12, 15, 32, 33 and returns a result based on the input on line 50.
1
​
2
int UnpackGetInfo_NEW(char *user_input,undefined4 *param_2,undefined4 *param_3)
3
​
4
{
5
undefined4 uVar1;
6
undefined4 uVar2;
7
undefined4 uVar3;
8
undefined4 *puVar4;
9
undefined4 *puVar5;
10
undefined4 *puVar6;
11
12
if (((*user_input != '\f') || (user_input[1] != 0x16)) || (*(short *)(user_input + 2) != 0x1f)) {
13
return 1;
14
}
15
puVar6 = (undefined4 *)(user_input + 8);
16
do {
17
puVar5 = puVar6;
18
puVar4 = param_2;
19
uVar1 = puVar5[1];
20
uVar2 = puVar5[2];
21
uVar3 = puVar5[3];
22
*puVar4 = *puVar5;
23
puVar4[1] = uVar1;
24
puVar4[2] = uVar2;
25
puVar6 = puVar5 + 4;
26
puVar4[3] = uVar3;
27
param_2 = puVar4 + 4;
28
} while (puVar6 != (undefined4 *)(user_input + 0xf8));
29
uVar1 = puVar5[5];
30
puVar4[4] = *puVar6;
31
puVar4[5] = uVar1;
32
if ((*(short *)(user_input + 0x110) == -0x7f7e) &&
33
(puVar6 = (undefined4 *)(user_input + 0x110), (user_input[0x112] & 1U) != 0)) {
34
do {
35
puVar5 = puVar6;
36
puVar4 = param_3;
37
uVar1 = puVar5[1];
38
uVar2 = puVar5[2];
39
uVar3 = puVar5[3];
40
*puVar4 = *puVar5;
41
puVar4[1] = uVar1;
42
puVar4[2] = uVar2;
43
puVar6 = puVar5 + 4;
44
puVar4[3] = uVar3;
45
param_3 = puVar4 + 4;
46
} while (puVar6 != (undefined4 *)(user_input + 0x1d0));
47
uVar1 = puVar5[5];
48
puVar4[4] = *puVar6;
49
puVar4[5] = uVar1;
50
return (uint)((user_input[0x112] & 0x10U) != 0) + 5;
51
}
52
return 0;
53
}
54
​
55
​
Copied!
This is a perfect time to breakout angr to create a valid input to hit line 50! The following code will create a 300 byte symbolic buffer and have angr solve the constraints required to pass each check in the unpacking function to yield all potential return results. We are intersted in the analysis path that reached the furthest part of the parsing function. The script below will print out each path end address and the required input to reach that path.
1
import angr
2
import angr.sim_options as so
3
import claripy
4
​
5
symbol = "UnpackGetInfo_NEW"
6
​
7
# Create a project with history tracking
8
p = angr.Project('/home/caffix/firmware/asus/RT-AC51U/ext_fw/squashfs-root/usr/sbin/asusdiscovery')
9
extras = {so.REVERSE_MEMORY_NAME_MAP, so.TRACK_ACTION_HISTORY}
10
​
11
# User input will be 300 symbolic bytes
12
user_arg = claripy.BVS("user_arg", 300*8)
13
​
14
# State starts at function address
15
start_addr = p.loader.find_symbol(symbol).rebased_addr
16
state = p.factory.blank_state(addr=start_addr, add_options=extras)
17
​
18
# Store symbolic user_input buffer
19
state.memory.store(0x100000, user_arg)
20
state.regs.a0 = 0x100000
21
​
22
# Run to exhaustion
23
simgr = p.factory.simgr(state)
24
simgr.explore()
25
​
26
# Print each path and the inputs required
27
for path in simgr.unconstrained:
28
print("{} : {}".format(path,hex([x for x in path.history.bbl_addrs][-1])))
29
u_input = path.solver.eval(user_arg, cast_to=bytes)
30
print(u_input)
31
​
Copied!
One of the outputs is shown below, and this input can then be sent back into the program through the above qemu command to validate that it passes the checks.
1
<SimState @ <BV32 reg_ra_51_32{UNINITIALIZED}>> : 0x401c4c
2
b'\x0c\x16\x1f\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x82\x80\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
3
​
4
### Running the input
5
$ printf '\x0c\x16\x1f\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x82\x80\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00' | qemu-mipsel -L . -E LD_PRELOAD=/lib/libdl.so.0:/main_hook.so ./usr/sbin/asusdiscovery
6
Ret val 1
7
​
Copied!
I've put each of these inputs into individual files for AFL to read from later.
1
$ ls afl_input/
2
test_case1 test_case2 test_case3 test_case4 test_case5
3
​
Copied!
Fuzzing the function
Using the AFL build process outlined here will provide AFL with qemu mode which will fuzz asusdiscovery with the script:
1
#!/bin/bash
2
​
3
export "QEMU_SET_ENV=LD_PRELOAD=/lib/libdl.so.0:/main_hook.so"
4
export "QEMU_LD_PREFIX=/home/caffix/firmware/asus/RT-AC51U/ext_fw/squashfs-root"
5
export "AFL_INST_LIBS=1"
6
#export "AFL_NO_FORKSRV=1"
7
​
8
BINARY="/home/caffix/firmware/asus/RT-AC51U/ext_fw/squashfs-root/usr/sbin/asusdiscovery"
9
​
10
afl-fuzz -i afl_input -o output -m none -Q $BINARY
11
​
Copied!
You will get some incredibly slow fuzzing at about 1-2 execution per second. The afl fork server is taking way to long to spawn off newly forked processes.
Adding the
AFL_NO_FORKSRV=1 will prevent AFL from creating a forkserver just before main and forking off new processes. For this type of hooking and emulation it runs much faster at about 85 executions per second:
We can do better... Specifically we can use Abiondo's fork of AFL that he describes his blog post here. Abiondo implemented an idea for QEMU that is quoted at speeding up the qemu emulation speed on a scale of 3 to 4 times. That should put us at 300 or 400 executions per second.
My idea was to move the instrumentation into the translated code by injecting a snippet of TCG IR at the beginning of every TB. This way, the instrumentation becomes part of the emulated program, so we don’t need to go back into the emulator at every block, and we can re-enable chaining.
Downloading and running the fork of AFL follows the exact same build process:
1
git clone https://github.com/abiondo/afl.git
2
cd afl
3
make
4
cd qemu_mode
5
export CPU_TARGET=mipsel
6
./build_qemu_support.sh
Copied!
Rerunning the previous fuzzing command script WITHOUT the
AFL_NO_FORKSRV environment variable produces some absolutely insane results:
Final fuzzing results
After about 24 hours of fuzzing, hardly any new paths were discovered. Doing some more static analysis on the parsing functions revealed very few spots in the functions for any potentially dangerous user input to corrupt anything.
1
$ cat output_fast/fuzzer_stats
2
start_time : 1555381507
3
last_update : 1555385229
4
fuzzer_pid : 61241
5
cycles_done : 272
6
execs_done : 8226287
7
execs_per_sec : 2055.33
8
paths_total : 85
9
paths_favored : 19
10
paths_found : 81
11
paths_imported : 0
12
max_depth : 6
13
cur_path : 49
14
pending_favs : 0
15
pending_total : 0
16
variable_paths : 0
17
stability : 100.00%
18
bitmap_cvg : 1.15%
19
unique_crashes : 0
20
unique_hangs : 0
21
last_path : 1555382334
22
last_crash : 0
23
last_hang : 0
24
execs_since_crash : 8226287
25
exec_timeout : 20
26
afl_banner : asusdiscovery
27
afl_version : 2.52b
28
target_mode : qemu
29
command_line : afl-fuzz -i afl_input -o output -m none -Q /home/caffix/firmware/asus/RT-AC51U/ext_fw/squashfs-root/usr/sbin/asusdiscovery
30
​
Copied!
Final thoughts
Over the course of using the LD_PRELOAD trick paired with jumping directly to a function I wanted to fuzz, I was able to save tons of time inside of GDB trying to see what code paths were valid. By using Abiondo's fork of AFL I was able to get execution times on par with AFL compiling code speeds. Getting thousands of executions per second doesn't generally happen when fuzzing applications in AFL's QEMU mode and I was happy to see 2000 plus executions per second.