Monday, December 23, 2013

Analyzing Compressed RAM in Mac OS X (and Linux)

The compressed RAM facility in Mac OS X Mavericks places a RAM compressor inline and in front of page swapping in the virtual memory implementation, favoring compressing pages of RAM over swapping them to disk.  In Mac OS X, page compression/decompression are handled by a custom, hand-optimized 64-bit assembler version of WKdm.

My Mavericks-compatible Python port of WKdm is done and I'm busy building the plugins to allow analysis of compressed RAM from Mac OS X memory dumps in Volatility.

+Andrew Case is helping with the Volatility integration, since we want support for decompressing pages to be as transparent as possible across various operating systems that implement compression. He's also tackling the Linux side, which uses a different set of compression algorithms.

I'll be releasing everything and talking about this project at the American Academy of Forensic Sciences (AAFS) meeting in Seattle in Feb 2014.  Hope to see you there.

Update (12/26/2013):

The Volatility plugins are now decompressing compressed RAM pages properly--currently, we're just dumping all the pages, but Andrew and I will integrate the decompression more tightly soon so that, e.g., process memory dumps for Mac OS X will also dump available compressed pages.

Update (12/27/2013):

I was curious if Apple's choice to do a custom assembler version of WKdm compression/decompression was worth it.  It definitely was--the hand-optimized assembler version, which eliminates all function calls and takes the WKdm stuff down to its bare essentials, smokes the original C version.  On a Core i7, with gcc and -O3:

Assembler timing:    268317 compression / decompression pairs per second.

C timing:    142150 compression / decompression pairs per second.

And, of course, there's the Python version we'll be using in the Volatility plugins:

Python timing: 391 compression / decompression pairs per second.


Update (1/2/2014):

Just for fun, I ported the Python version to go--in about an hour.

Go timing: 59880 compression / decompression pairs per second.  Basically, go rocks, and is my new language of choice.  If you haven't checked it out, do.

Update (1/23/2014):

Getting the Volatility plugins to handle decompression of pages for address spaces of individual processes took some effort, but that stuff is now working.  More detail on everything coming soon once I do some cleanup.

Update (4/19/2014):

Our paper describing this work, entitled "In Lieu of Swap: Analyzing Compressed RAM in Mac OS X and Linux" was accepted for publication at DFRWS 2014 in Denver.  See you in August!

No comments: