16 October 2017

This article is part of a series of blog posts. We recommend that you start at the beginning. Alternatively, scroll to the bottom of this article to navigate through the whole series.

We’re releasing a gdb plugin for analysing ptmalloc2. This plugin is essentially a fork from an older version of cloudburst’s libheap [1], but with some extras.

We are aware that libheap, gef [2] and pwndbg [3] all contain their own good code to analyse ptmalloc2 structures. libptmalloc [4] was designed to be more consistent with individual gdb commands such as those found in libtalloc [5] and libdlmalloc [6]. This allows us to add support for callbacks, which lets us annotate ptmalloc2 chunk contents with additional information about Cisco ASA specific structures.

Note that libptmalloc has been heavily tested with recent 64-bit Cisco ASA versions (both ASA5500-X series and GNS3) that use glibc’s ptmalloc2-based allocator.

Some old 64-bit ASA versions actually use dlmalloc, so you can refer to our previous blog post for analysing them.

You can refer to this table [7] for a list of the versions using ptmalloc2.

libptmalloc commands

As there is exhaustive information about ptmalloc2 available online, we will simply list the commands we’ve added to libptmalloc. We have retained the majority of libheap functionality and therefore don’t document it. Similarly to libdlmalloc, libptmalloc is completely integrated with asadbg, detailed in a previous blog post.

pthelp

This command shows other gdb commands. Each of the commands supports the -h option which allows you to obtain more detailed usage instructions.

(gdb) pthelp
[libptmalloc] ptmalloc commands for gdb
[libptmalloc] ptchunk -v -x -c   : show chunk contents (-v for verbose, -x for data dump)
[libptmalloc] ptsearch             : search heap for hex value or address
[libptmalloc] ptarena                : print mstate struct. caches address after first use
[libptmalloc] pthelp

ptarena

We will show this command first as it allows us to find a chunk to analyse later using other commands.

(gdb) ptarena -h
[libptmalloc] usage: ptarena [-v] [-f] [-x] [-c ] 
[libptmalloc]   a ptmalloc mstate struct. Optional with cached mstate
[libptmalloc] -v      use verbose output (multiples for more verbosity)
[libptmalloc] -l      list arenas only
[libptmalloc] NOTE:  Last defined mstate will be cached for future use

We can take a look at an arena, by pulling the main_arena symbol from glibc.so:

(gdb) ptarena 0x7ffff4c9b620
struct malloc_mstate {
mutex          = 0x0
flags          = 0x0
fastbinY[0] = 0x0
fastbinY[1] = 0x0
fastbinY[2] = 0x0
fastbinY[3] = 0x0
fastbinY[4] = 0x0
fastbinY[5] = 0x55555e8aecb0
fastbinY[6] = 0x55555e8aed20
fastbinY[7] = 0x0
fastbinY[8] = 0x0
fastbinY[9] = 0x0
top            = 0x55555ea644d0
last_remainder = 0x55555e8b3020
bin[0]:     = 0x55555ea62860, 0x55555e979ad0
bin[1]:     = 0x55555e920420, 0x55555e6363d0
bin[2]:     = 0x55555e641010, 0x55555e63cdc0
bin[3]:     = 0x7ffff4c9b6a8, 0x7ffff4c9b6a8
bin[4]:     = 0x7ffff4c9b6b8, 0x7ffff4c9b6b8
bin[5]:     = 0x55555ea62a70, 0x55555ea62a70
bin[6]:     = 0x7ffff4c9b6d8, 0x7ffff4c9b6d8
bin[7]:     = 0x7ffff4c9b6e8, 0x7ffff4c9b6e8
bin[8]:     = 0x7ffff4c9b6f8, 0x7ffff4c9b6f8
bin[9]:     = 0x55555e9254a0, 0x55555e9254a0
bin[10]:    = 0x7ffff4c9b718, 0x7ffff4c9b718
bin[11]:    = 0x55555e78a7f0, 0x55555e78a7f0
bin[12]:    = 0x7ffff4c9b738, 0x7ffff4c9b738
bin[13]:    = 0x7ffff4c9b748, 0x7ffff4c9b748
bin[14]:    = 0x55555e926c70, 0x55555e926c70
bin[15]:    = 0x7ffff4c9b768, 0x7ffff4c9b768
[SNIP]
bin[51]:    = 0x7ffff4c9b9a8, 0x7ffff4c9b9a8
bin[52]:    = 0x55555e988db0, 0x55555e988db0
bin[53]:    = 0x7ffff4c9b9c8, 0x7ffff4c9b9c8
bin[54]:    = 0x7ffff4c9b9d8, 0x7ffff4c9b9d8
bin[55]:    = 0x7ffff4c9b9e8, 0x7ffff4c9b9e8
bin[56]:    = 0x7ffff4c9b9f8, 0x7ffff4c9b9f8
bin[57]:    = 0x7ffff4c9ba08, 0x7ffff4c9ba08
bin[58]:    = 0x7ffff4c9ba18, 0x7ffff4c9ba18
bin[59]:    = 0x7ffff4c9ba28, 0x7ffff4c9ba28
bin[60]:    = 0x7ffff4c9ba38, 0x7ffff4c9ba38
bin[61]:    = 0x7ffff4c9ba48, 0x7ffff4c9ba48
bin[62]:    = 0x7ffff4c9ba58, 0x7ffff4c9ba58
bin[63]:    = 0x7ffff4c9ba68, 0x7ffff4c9ba68
bin[64]:    = 0x55555e94f1c0, 0x55555ea1e2b0
bin[65]:    = 0x7ffff4c9ba88, 0x7ffff4c9ba88
bin[66]:    = 0x7ffff4c9ba98, 0x7ffff4c9ba98
bin[67]:    = 0x7ffff4c9baa8, 0x7ffff4c9baa8
bin[68]:    = 0x7ffff4c9bab8, 0x7ffff4c9bab8
bin[69]:    = 0x7ffff4c9bac8, 0x7ffff4c9bac8
bin[70]:    = 0x7ffff4c9bad8, 0x7ffff4c9bad8
bin[71]:    = 0x7ffff4c9bae8, 0x7ffff4c9bae8
bin[72]:    = 0x55555e987f10, 0x55555e94f990
bin[73]:    = 0x7ffff4c9bb08, 0x7ffff4c9bb08
bin[74]:    = 0x7ffff4c9bb18, 0x7ffff4c9bb18
bin[75]:    = 0x7ffff4c9bb28, 0x7ffff4c9bb28
bin[76]:    = 0x7ffff4c9bb38, 0x7ffff4c9bb38
bin[77]:    = 0x7ffff4c9bb48, 0x7ffff4c9bb48
[SNIP]
bin[126]:   = 0x7ffff4c9be58, 0x7ffff4c9be58
binmap[0]   = 0x48947c
binmap[1]   = 0x200004
binmap[2]   = 0x202
binmap[3]   = 0x0
next           = 0x7fffa4000020
next_free      = 0x0
system_mem     = 0x444000
max_system_mem = 0x444000

We can dump all of the other arenas referenced by this arena:

(gdb) ptarena 0x7ffff4c9b620 -l
Arena(s) found:
     arena @ 0x7ffff4c9b620
     arena @ 0x7fffa4000020
     arena @ 0x7fffb0000020
     arena @ 0x7fffac000020
     arena @ 0x7fffb8000020
     arena @ 0x7fffb4000020
     arena @ 0x7fffc0000020
     arena @ 0x7fffbc000020
     arena @ 0x7fffc8000020
     arena @ 0x7fffc4000020
     arena @ 0x7fffcc000020
     arena @ 0x7fffd0000020
     arena @ 0x7fffd8000020

ptchunk

The ptchunk command is used to show information related to a chunk.

(gdb) ptchunk -h
[libptmalloc] usage: ptchunk [-v] [-f] [-x] [-c ] 
[libptmalloc]   a ptmalloc chunk header
[libptmalloc] -v      use verbose output (multiples for more verbosity)
[libptmalloc] -f      use  explicitly, rather than be smart
[libptmalloc] -x      hexdump the chunk contents
[libptmalloc] -m      max bytes to dump with -x
[libptmalloc] -c      number of chunks to print
[libptmalloc] -d      debug and force printing stuff
[libptmalloc] Flag legend: P=PREV_INUSE, M=MMAPPED, N=NON_MAIN_ARENA

We select a chunk from the earlier ptarena output and take a look:

(gdb) ptchunk -c 5 0x55555e988db0
0x55555e988db0 F sz:0x00350 fl:--P free_pc:0x00000000,-
0x55555e989100 M sz:0x04460 fl:--- alloc_pc:0x555558672bfc,-
0x55555e98d560 M sz:0x00310 fl:--P alloc_pc:0x555558672c75,-
0x55555e98d870 M sz:0x00130 fl:--P alloc_pc:0x5555563269af,-
0x55555e98d9a0 M sz:0x00070 fl:--P alloc_pc:0x555556338c75,-

We see that the output includes alloc_pc and free_pc information from a mempool callback we have registered with ptcallback, which we will describe later.

As expected, you can use -v to get additional structure information:

(gdb) ptchunk -v 0x55555e988db0
struct malloc_chunk @ 0x55555e988db0 {
prev_size    = 0x0
size         = 0x350 (PREV_INUSE)
fd           = 0x7ffff4c9b9b8
bk           = 0x7ffff4c9b9b8

We can look at one of the in-use chunks:

(gdb) ptchunk -v 0x55555e989100
struct malloc_chunk @ 0x55555e989100 {
prev_size    = 0x350
size         = 0x4460

You can see the contents of chunks using -x:

(gdb) ptchunk -x 0x55555e98d9a0
0x55555e98d9a0 M sz:0x00070 fl:--P alloc_pc:0x555556338c75,-
0x60 bytes of chunk data:
0x55555e98d9b0: 0xa11c0123 0x00000030 0x00010000 0x00000000
0x55555e98d9c0: 0x5e988be0 0x00005555 0x5e98dbd0 0x00005555
0x55555e98d9d0: 0x56338c75 0x00005555 0x00000000 0x00000000
0x55555e98d9e0: 0x00000000 0x00000000 0x00000000 0x00000000
0x55555e98d9f0: 0x00000000 0x00000000 0x00000000 0x00000000
0x55555e98da00: 0x00000000 0x00000000 0x00006168 0x00000000

ptsearch

This is a command that lets us search for values in each chunk across all arenas. Naturally, it can be quite slow, especially over serial (which is the case when analysing real Cisco ASA devices). It relies on a cached arena address (set with ptarena command) and it will parse all arenas linked to the one that is cached. Here is an example of searching for the mh_magic value 0xa11c0123:

(gdb) ptsearch 0xa11c0123
[libptmalloc] Handling arena @ 0x7fffa4000020
[libptmalloc] chunk with zero size detected at 0x7fffa4021000
[libptmalloc] sz=0x0 detected at 0x7fffa4021000, assuming end of heap
[libptmalloc] 0xa11c0123 found in chunk at 0x7fffa40008b0
[libptmalloc] 0xa11c0123 found in chunk at 0x7fffa4000930
[libptmalloc] 0xa11c0123 found in chunk at 0x7fffa40009a0
[libptmalloc] 0xa11c0123 found in chunk at 0x7fffa4000a10
[libptmalloc] 0xa11c0123 found in chunk at 0x7fffa4000a60
[libptmalloc] 0xa11c0123 found in chunk at 0x7fffa4000ab0
[libptmalloc] Handling arena @ 0x7fffb0000020
[libptmalloc] chunk with zero size detected at 0x7fffb0021000
[libptmalloc] sz=0x0 detected at 0x7fffb0021000, assuming end of heap
[libptmalloc] 0xa11c0123 found in chunk at 0x7fffb00008b0
[libptmalloc] 0xa11c0123 found in chunk at 0x7fffb0000930
[libptmalloc] 0xa11c0123 found in chunk at 0x7fffb0000990
[libptmalloc] 0xa11c0123 found in chunk at 0x7fffb0000a30
[libptmalloc] 0xa11c0123 found in chunk at 0x7fffb0000d00
[libptmalloc] 0xa11c0123 found in chunk at 0x7fffb0000d50
[libptmalloc] 0xa11c0123 found in chunk at 0x7fffb0000e20
[libptmalloc] 0xa11c0123 found in chunk at 0x7fffb0000f00
[libptmalloc] 0xa11c0123 found in chunk at 0x7fffb0000f70
[libptmalloc] 0xa11c0123 found in chunk at 0x7fffb0001330
[libptmalloc] 0xa11c0123 found in chunk at 0x7fffb0003370
[libptmalloc] 0xa11c0123 found in chunk at 0x7fffb00053b0
[libptmalloc] Handling arena @ 0x7fffac000020
[...]

ptcallback

This is a command that allows you to register a callback that can help annotate additional structures that might be contained inside of a chunk. In our case we use this to typically show Cisco ASA mempool headers.

(gdb) ptcallback -h
[libptmalloc] usage: ptcallback 
[libptmalloc] disable          temporarily disable the registered callback
[libptmalloc] enable           enable the registered callback
[libptmalloc] status           check if a callback is registered
[libptmalloc] clear            forget the registered callback
[libptmalloc] register   use a global function  as callback
[libptmalloc] register    use a global function  as callback from

An example of using most of the functionality is shown below. We first register the callback but disable it:

(gdb) ptcallback status
[libptmalloc] a callback is not registered
(gdb) ptcallback register mpcallback libmempool/libmempool
[libmempool] loaded
[libptmalloc] mpcallback registered as callback
(gdb) ptcallback status
[libptmalloc] a callback is registered and enabled
(gdb) ptcallback disable
[libptmalloc] callback disabled
(gdb) ptcallback status
[libptmalloc] a callback is registered and disabled

With the callback disabled, it only shows the ptmalloc chunk header:

(gdb) ptchunk -v 0x55555e98d9a0
struct malloc_chunk @ 0x55555e98d9a0 {
prev_size    = 0xa11ccdef
size         = 0x70 (PREV_INUSE)

Now we enable the callback and again display metadata. We see it shows the mp_header in addition to the ptmalloc chunk header:

(gdb) ptcallback enable
[libptmalloc] callback enabled
(gdb) ptchunk -v 0x55555e98d9a0
struct malloc_chunk @ 0x55555e98d9a0 {
prev_size    = 0xa11ccdef
size         = 0x70 (PREV_INUSE)
struct mp_header @ 0x55555e98d9b0 {
mh_magic      = 0xa11c0123
mh_len        = 0x30
mh_refcount   = 0x10000
mh_unused     = 0x0
mh_fd_link    = 0x55555e988be0 (OK)
mh_bk_link    = 0x55555e98dbd0 (OK)
alloc_pc      = 0x555556338c75 (-)
free_pc       = 0x0 (-)

We can completely clear the callback:

(gdb) ptcallback clear
[libptmalloc] callback cleared
(gdb) ptchunk -v 0x55555e98d9a0
struct malloc_chunk @ 0x55555e98d9a0 {
prev_size   = 0xa11ccdef
size        = 0x70 (PREV_INUSE)

Exploiting ptmalloc chunk corruption on Cisco ASA

In general, exploitation of ptmalloc2 on the Cisco ASA is similar to other platforms.

The main thing to note is that the ptmalloc2 algorithm used appears to be a mostly unmodified version from glibc.so, it does contain safe unlinking for free chunk coalescing. What it means for an exploitation scenario is that the path of least resistance is to instead target the mh_fd_link and mh_bk_link entries of the mempool headers, which are not subject to safe unlinking.

Conclusion

We’ve provided a brief overview of the libptmalloc plugin and the commands that it exposes, and how they might be useful when debugging 64-bit Cisco ASA devices. The main highlights of the tool are the ability to analyse one or more chunks in sequence (which can aid in feng shui analysis) and the ability to register callbacks that can be used to augment what information is shown when detailing a chunk.

We would appreciate any feedback or corrections. You can test out the libptmalloc code and feel free to send pull requests for any issues you have. The tool is not perfect, so don’t be surprised if you run into bugs. If you would like to contact us we can be reached by email or twitter: aaron(dot)adams(at)nccgroup(dot)trust / @fidgetingbits and cedric(dot)halbronn(at)nccgroup(dot)trust / @saidelike.