Failure Observation Engine (FOE) 2.1 README ===== License ===== =================== See LICENSE.txt ===== Change Log ===== ====================== See NEWS.txt ===== Quick Start ======== ========================== Because fuzzing can fill temporary directories, put the target application in an unusable state, or trigger other operating-system-level bugs, we recommend that FOE be used in a virtual machine. Run FOE-2.1-setup.exe in a virtual machine to install FOE 2.1. The installer should detect and attempt to download prerequisites and configure your environment appropriately. ===== Running FOE ===== ======================= 1) Click the FOE2 item in the Windows Start menu. 2) Run foe2.py 3) Run tools\quickstats.py to check fuzzing progress when you wish. ===== How it works ===== ======================== When a campaign starts, FOE will gather available seed files and create scorable sets: 1) The seed files themselves 2) The fuzz percent ranges for each seed file Each interval of a campaign will choose a seed file, and then for that file, it will choose an percent range to mangle the file. After mangling the file, FOE will launch the target application, using the configured command line to have it parse the fuzzed file. If the "winrun" runner is compatible with the current platform, this is accomplished by preloading a crash-intercepting hook into the target application's process space. This allows crash detection without relying on a debugger. The "nullrun" runner simply runs each invocation through the debugger (cdb). When a crash is detected, it is then verified using a combination of cdb and the Microsoft !exploitable debugger extension. If the crash is determined to be unique (by the chain of !exploitable crash hashes), then some additional analysis steps are taken: 1) A !exploitable report is created for each continuable exception. 2) If configured to do so, FOE will create a minimized test case. 3) The seed file and percent range that were used to fuzz are scored Seed files that produce more crashes are given a preference over less- productive files, and for each seed file, the mangling percent ranges that are more productive are also given preference. These scoring features together minimize the amount of knowledge required to perform an effective fuzzing campaign. ===== Analyzing results ===== ============================= .\results\<campaignid>\ +- <configname>.yaml +- version.txt +- <SEVERITY>/ +- <hash_1>/ +- minimizer_log.txt +- sf_<seedfile_hash>.<ext> +- sf_<seedfile_hash>-<iteration>-<EFA>.<ext> +- sf_<seedfile_hash>-<iteration>-<EFA>-<SEVERITY>.<ext>.msec +- sf_<seedfile_hash>-<iteration>-<EFA>-minimized.<ext> +- sf_<seedfile_hash>-<iteration>.<ext>.e<n>.msec +- <hash_2>/ +- ... +- <hash_n>/ <configname>.yaml This is a copy of the config file used for this run. It is stored for historical purposes ("Which options did I use for that run?"). version.txt This file stores the version of FOE that was used for fuzzing. <SEVERITY> This is the "Exploitability Classification" assigned to the crash by !exploitable. Values can be EXPLOITABLE, PROBABLY_EXPLOITABLE, UNKNOWN, or PROBABLY_NOT_EXPLOITABLE. For crashes that include multiple exceptions, the highest exploitability of any of the exceptions is used for this directory. Be aware that !exploitable has limitations and only provides a rough (possibly false-positive) assesment of a crash. More information on !exploitable can be found here: http://msecdbg.codeplex.com/ http://blogs.technet.com/b/srd/archive/2009/04/08/the-history-of-the-exploitable-crash-analyzer.aspx <hash_n> This is the hash in Major.Minor form provided by !exploitable. minimizer_log.txt This is the log file that was produced during crash minimization. sf_<seedfile_hash>.<ext> This is the original file (pre-fuzz). This is provided as a convenient "diff" source. sf_<seedfile_hash>-<iteration>-<EFA>.<ext> This is the fuzzed file that caused the crash. <EFA> is the exception faulting address, as reported by !exploitable. sf_<seedfile_hash>-<iteration>-<EFA>-<SEVERITY>.<ext>.msec This is the cdb text output from the crash, which includes output from the !exploitable tool. sf_<seedfile_hash>-<iteration>-<EFA>-minimized.<ext> This is the minimized version of the crashing test case. It is the "least different" version of the original fuzzed file that caused a specific crash (hash). sf_<seedfile_hash>-<iteration>.<ext>.e<n>.msec This is the cdb output for an exception that is continued <n> number of times. One file is provided for each continued exception until an uncontinuable exception is encountered, or the handled exception limit has been reached, or the target application proceeds without encountering another exception. ===== Fuzzing on your own ===== =============================== Once you are comfortable with FOE's default ImageMagick fuzz run, you can try fuzzing an application of your choice. The first step is to place seed files into the FOE seedfiles directory. These are the files that will be mangled and opened by the target application. Next modify the foe.yaml file to suit your needs. The foe.yaml file is documented to describe what each of the features mean. The important parts to modify are: campaign: id: This field is used in determining the fuzzing campaign, and subsequently, where the results should be stored. This should probably be the target application name and version. campaign: use_buttonclicker: When fuzzing a GUI application, the FOE button clicker can increase throughput and code coverage. Note that the button clicker is not configurable, but rather it has a built-in heuristic for determining which buttons to click. target: program: This is the full path to the target application that you wish to fuzz. target: cmdline_template: This specifies the commandline syntax for invoking the target application. runner: runtimeout: This value specifies how long FOE should wait before terminating the application and moving on to the next iteration. Note that this setting only applies to the "winrun" runner (32-bit Windows XP and Server 2003 systems). debugger: runtimeout: This value specifies how long FOE should allow the target application to run when it is invoked from the debugger. On platforms that use the "null" runner (64-bit Windows or Windows Vista or newer), this is the only timeout value that is used. FOE periodically saves state of a fuzzing campaign, so it will by default continue a cached campaign if foe.yaml has not been modified. To clear the FOE cached state, run: tools\clean_foe.py For additional options, run: tools\clean_foe.py --help ===== Digging deeper into results ===== ======================================= When FOE has produced results, you may wish to perform some additional steps. Finding interesting crashes: With some target applications, FOE may produce too many uniquely-crashing test cases to investigate manually in a reasonable amount of time. We have provided a script called drillresults.py to pick out crashes that are most likely to be exploitable and list those cases in a ranked order (most exploitable first). To run this script, run: tools\drillresults.py For command-line usage, run: tools\drillresults.py --help Reproducing crashes: The tools\repro.py script can be used to reproduce a crash by running it in the same manner that FOE did. For command-line usage, run: tools\repro.py --help Comparing zip-based files: The tools\zipdiff.py script can be used to compare zip-based files. For command-line usage, run: tools\zipdiff.py --help Minimization to string: Say you have a crashing test case, but you really need to get it to a proof-of-concept exploit. The problem is when you load the crash into your debugger you can't easily tell which registers, stack values, or memory locations are under your control. But what if you could change the crashing test case so that it had only the bytes required to cause that crash, and the rest were all masked out with a fixed value, say "x" (0x78)? Then you'd know that if you saw EIP=0x78787878, you may already be a winner. The minimize-to-string option does just that. To get command-line usage of the minimizer, run: tools\minimize.py --help To minimize a crashing testcase to the Metasploit string pattern, run: tools\minimize.py --stringmode <crashing_testcase> When minimizing to the Metasploit pattern, FOE will use the resulting byte map to create an additional minimized file that uses a string of 'x' characters. Note that this file is not guaranteed to produce the same crash as the original string minimization. Metasploit pattern enumeration: Especially with larger files, you may notice that the Metasploit pattern repeats several times over the length of a Metasploit-minimized crasher. Given any particular dword, it may not be obvious which instance is the one that you are dealing with. This is where the tools\mtsp_enum.py script comes in handy. For example, let's say that you have a crasher.doc were EIP = "Aa0A" If you run: tools\mtsp_enum.py Aa0A crasher.doc You will end up with a file called crasher-enum.doc. With this file, every instance of the byte pattern "Aa0A" will be replaced with a unique, incrementing replacement. For example, "0a0A", "1a0A", "2a0A", etc. Now when you open crasher-enum.doc, you could for example get EIP = "5a0A". If you search for that pattern in the file, there should be only once instance of it. Note that you can use a search pattern of any length and you can also search for hex values. For example: "\x01\x02\x03\x04" ===== Included Fuzzing Strategies ===== ======================================= bytemut: replace bytes with random values swap: swap adjacent bytes wave: cycle through every possible single-byte value, sequentially drop: removes one byte from the file for each position in the file insert: inserts a random byte for each position in the file truncate: truncates bytes from the end of the file crmut: replace carriage return bytes with random values crlfmut: replace carriage return and linefeed bytes with random values nullmut: replace null bytes with random values verify: do not mutate file. Used for verifying crashing testcases range_list: byte ranges to be fuzzed. One range per line, hex or decimal ===== Verifying crashing results ====== ======================================= FOE can be used to verify crashing test cases. This can be useful for when a new version of an application is released or if you are the developer and you want to see how many uniquely-crashing test cases disappear when you fix a bug. To perform a verfification campaign: 1) Run tools\copycrashers.py to collect all of the crashing cases from a campaign. By default it will copy all of the uniquely-crashing test cases to the "seedfiles" directory, which should be empty. 2) Modify configs\foe.yaml to use the "verify" fuzzer and also specify a new campaign ID. When you run FOE, it will run each case with the target application, and cases that still crash will be placed in the results directory for the new campaign. ===== Manually Installing FOE ===== =================================== If you have installed FOE using the installer, you can skip this section. To install FOE manually, you will need the following prerequisites: - Windows XP or Server 2003 32-bit is recommended to allow exception hooking (winrun) Other Windows versions will use debugger mode (nullrun) - Python 2.7 http://www.python.org/download/releases/2.7.5/ - SciPy http://sourceforge.net/projects/scipy/files/scipy/0.10.1/scipy-0.10.1-win32-superpack-python2.7.exe/download - NumPy http://sourceforge.net/projects/numpy/files/NumPy/1.6.1/numpy-1.6.1-win32-superpack-python2.7.exe/download - PyYAML http://pyyaml.org/download/pyyaml/PyYAML-3.10.win32-py2.7.exe - pywin32 http://sourceforge.net/projects/pywin32/files/pywin32/Build%20218/pywin32-218.win32-py2.7.exe/download - Python WMI https://pypi.python.org/packages/any/W/WMI/WMI-1.4.9.win32.exe - Debugging Tools for Windows http://www.microsoft.com/whdc/devtools/debugging/default.mspx Set up symbols, if so desired. - Microsoft !exploitable Copy the !exploitable dll (msec.dll) to winext directory. (probably C:\Program Files\Debugging Tools for Windows (x86)\winext) http://msecdbg.codeplex.com/ - Add debugging tools (specifically cdb.exe) to your PATH. (probably C:\Program Files\Debugging Tools for Windows (x86)\) - Copy the foe.yaml config file from configs\examples\ to a configs and modify as necessary. - Copy seed files to the seedfiles directory.