Failure Observation Engine (FOE) 2.1 README

License

See LICENSE.txt

Change Log

See NEWS.txt

Quick Start

Because fuzzing can fill temporary directories, put the target application  in an unusable state, or trigger other operating-system-level bugs, we recommend that FOE be used in a virtual machine.

Run FOE-2.1-setup.exe in a virtual machine to install FOE 2.1.

The installer should detect and attempt to download prerequisites and configure your environment appropriately.

Running FOE

  1. Click the FOE2 item in the Windows Start menu.
  2. Run foe2.py
  3. Run tools\quickstats.py to check fuzzing progress when you wish.

 

How it works

When a campaign starts, FOE will gather available seed files and create scorable sets:

  1. The seed files themselves
  2. The fuzz percent ranges for each seed file


Each interval of a campaign will choose a seed file, and then for that file,
it will choose an percent range to mangle the file. After mangling the file,
FOE will launch the target application, using the configured command line to
have it parse the fuzzed file. If the "winrun" runner is compatible with the
current platform, this is accomplished by preloading a crash-intercepting hook
into the target application's process space. This allows crash detection without
relying on a debugger. The "nullrun" runner simply runs each invocation
through the debugger (cdb).

When a crash is detected, it is then verified using a combination of cdb and
the Microsoft !exploitable debugger extension. If the crash is determined to
be unique (by the chain of !exploitable crash hashes), then some additional
analysis steps are taken:
1) A !exploitable report is created for each continuable exception.
2) If configured to do so, FOE will create a minimized test case.
3) The seed file and percent range that were used to fuzz are scored

Seed files that produce more crashes are given a preference over less-
productive files, and for each seed file, the mangling percent ranges that
are more productive are also given preference. These scoring features together
minimize the amount of knowledge required to perform an effective fuzzing
campaign.

Analyzing results


.\results\<campaignid>\
  +- <configname>.yaml
  +- version.txt
  +- <SEVERITY>/
     +- <hash_1>/
        +- minimizer_log.txt
        +- sf_<seedfile_hash>.<ext>
        +- sf_<seedfile_hash>-<iteration>-<EFA>.<ext>
        +- sf_<seedfile_hash>-<iteration>-<EFA>-<SEVERITY>.<ext>.msec
        +- sf_<seedfile_hash>-<iteration>-<EFA>-minimized.<ext>
        +- sf_<seedfile_hash>-<iteration>.<ext>.e<n>.msec
     +- <hash_2>/
     +- ...
     +- <hash_n>/


<configname>.yaml
This is a copy of the config file used for this run. It is stored for
historical purposes ("Which options did I use for that run?").

version.txt
This file stores the version of FOE that was used for fuzzing.

<SEVERITY>
This is the "Exploitability Classification" assigned to the crash by
!exploitable. Values can be EXPLOITABLE, PROBABLY_EXPLOITABLE, UNKNOWN, or
PROBABLY_NOT_EXPLOITABLE. For crashes that include multiple exceptions,
the highest exploitability of any of the exceptions is used for this
directory. Be aware that !exploitable has limitations and only provides
a rough (possibly false-positive) assesment of a crash.
More information on !exploitable can be found here:

http://msecdbg.codeplex.com/
http://blogs.technet.com/b/srd/archive/2009/04/08/the-history-of-the-exploitable-crash-analyzer.aspx

<hash_n>
This is the hash in Major.Minor form provided by !exploitable.

minimizer_log.txt
This is the log file that was produced during crash minimization.

sf_<seedfile_hash>.<ext>
This is the original file (pre-fuzz). This is provided as a convenient
"diff" source.

sf_<seedfile_hash>-<iteration>-<EFA>.<ext>
This is the fuzzed file that caused the crash. <EFA> is the exception faulting
address, as reported by !exploitable.

sf_<seedfile_hash>-<iteration>-<EFA>-<SEVERITY>.<ext>.msec
This is the cdb text output from the crash, which includes output from the
!exploitable tool.

sf_<seedfile_hash>-<iteration>-<EFA>-minimized.<ext>
This is the minimized version of the crashing test case. It is the "least
different" version of the original fuzzed file that caused a specific
crash (hash).

sf_<seedfile_hash>-<iteration>.<ext>.e<n>.msec
This is the cdb output for an exception that is continued <n> number of times.
One file is provided for each continued exception until an uncontinuable
exception is encountered, or the handled exception limit has been reached, or
the target application proceeds without encountering another exception.

Fuzzing on your own


Once you are comfortable with FOE's default ImageMagick fuzz run, you can
try fuzzing an application of your choice. The first step is to place seed
files into the FOE seedfiles directory. These are the files that will be
mangled and opened by the target application. Next modify the foe.yaml file
to suit your needs.  The foe.yaml file is documented to describe what each
of the features mean. The important parts to modify are:

campaign: id:
    This field is used in determining the fuzzing campaign, and subsequently,
    where the results should be stored. This should probably be the target
    application name and version.

campaign: use_buttonclicker:
    When fuzzing a GUI application, the FOE button clicker can increase
    throughput and code coverage. Note that the button clicker is not
    configurable, but rather it has a built-in heuristic for determining which
    buttons to click.

target: program:
    This is the full path to the target application that you wish to fuzz.

target: cmdline_template:
    This specifies the commandline syntax for invoking the target application.

runner: runtimeout:
    This value specifies how long FOE should wait before terminating the
    application and moving on to the next iteration.
    Note that this setting only applies to the "winrun" runner (32-bit Windows
    XP and Server 2003 systems).

debugger: runtimeout:
    This value specifies how long FOE should allow the target application to
    run when it is invoked from the debugger. On platforms that use the "null"
    runner (64-bit Windows or Windows Vista or newer), this is the only
    timeout value that is used.

FOE periodically saves state of a fuzzing campaign, so it will by default
continue a cached campaign if foe.yaml has not been modified.
To clear the FOE cached state, run:
tools\clean_foe.py
For additional options, run:
tools\clean_foe.py --help

Digging deeper into results

When FOE has produced results, you may wish to perform some additional steps.

Finding interesting crashes:

With some target applications, FOE may produce too many uniquely-crashing test
cases to investigate manually in a reasonable amount of time. We have provided
a script called drillresults.py to pick out crashes that are most likely to be
exploitable and list those cases in a ranked order (most exploitable first).

To run this script, run:
tools\drillresults.py
For command-line usage, run:
tools\drillresults.py --help

Reproducing crashes:

The tools\repro.py script can be used to reproduce a crash by running it in
the same manner that FOE did.
For command-line usage, run:
tools\repro.py --help

Comparing zip-based files:

The tools\zipdiff.py script can be used to compare zip-based files.
For command-line usage, run:
tools\zipdiff.py --help

Minimization to string:

Say you have a crashing test case, but you really need to get it to a
proof-of-concept exploit. The problem is when you load the crash into your
debugger you can't easily tell which registers, stack values, or memory
locations are under your control. But what if you could change the crashing
test case so that it had only the bytes required to cause that crash, and the
rest were all masked out with a fixed value, say "x" (0x78)? Then you'd know
that if you saw EIP=0x78787878, you may already be a winner. The
minimize-to-string option does just that.
To get command-line usage of the minimizer, run:
tools\minimize.py --help

To minimize a crashing testcase to the Metasploit string pattern, run:
tools\minimize.py --stringmode <crashing_testcase>

When minimizing to the Metasploit pattern, FOE will use the resulting byte map
to create an additional minimized file that uses a string of 'x' characters.
Note that this file is not guaranteed to produce the same crash as the
original string minimization.

Metasploit pattern enumeration:

Especially with larger files, you may notice that the Metasploit pattern
repeats several times over the length of a Metasploit-minimized crasher.
Given any particular dword, it may not be obvious which instance is the one
that you are dealing with. This is where the tools\mtsp_enum.py script comes
in handy. For example, let's say that you have a crasher.doc were EIP = "Aa0A"
If you run: tools\mtsp_enum.py Aa0A crasher.doc
You will end up with a file called crasher-enum.doc.  With this file, every
instance of the byte pattern "Aa0A" will be replaced with a unique,
incrementing replacement. For example, "0a0A", "1a0A", "2a0A", etc. Now when
you open crasher-enum.doc, you could for example get EIP = "5a0A". If you
search for that pattern in the file, there should be only once instance of it.
Note that you can use a search pattern of any length and you can also search
for hex values. For example: "\x01\x02\x03\x04"

Included Fuzzing Strategies


bytemut: replace bytes with random values
swap: swap adjacent bytes
wave: cycle through every possible single-byte value, sequentially
drop: removes one byte from the file for each position in the file
insert: inserts a random byte for each position in the file
truncate: truncates bytes from the end of the file
crmut: replace carriage return bytes with random values
crlfmut: replace carriage return and linefeed bytes with random values
nullmut: replace null bytes with random values
verify: do not mutate file. Used for verifying crashing testcases
range_list: byte ranges to be fuzzed. One range per line, hex or decimal

Verifying crashing results


FOE can be used to verify crashing test cases. This can be useful for
when a new version of an application is released or if you are the
developer and you want to see how many uniquely-crashing test cases
disappear when you fix a bug. To perform a verfification campaign:

1) Run tools\copycrashers.py to collect all of the crashing cases
from a campaign. By default it will copy all of the uniquely-crashing
test cases to the "seedfiles" directory, which should be empty.
2) Modify configs\foe.yaml to use the "verify" fuzzer and also specify
a new campaign ID.

When you run FOE, it will run each case with the target application,
and cases that still crash will be placed in the results directory for
the new campaign.

Manually Installing FOE


If you have installed FOE using the installer, you can skip this section.
To install FOE manually, you will need the following prerequisites:

- Windows XP or Server 2003 32-bit is recommended to allow exception hooking
  (winrun)
  Other Windows versions will use debugger mode (nullrun)

- Python 2.7
http://www.python.org/download/releases/2.7.5/

- SciPy
http://sourceforge.net/projects/scipy/files/scipy/0.10.1/scipy-0.10.1-win32-superpack-python2.7.exe/download

- NumPy
http://sourceforge.net/projects/numpy/files/NumPy/1.6.1/numpy-1.6.1-win32-superpack-python2.7.exe/download

- PyYAML
http://pyyaml.org/download/pyyaml/PyYAML-3.10.win32-py2.7.exe

- pywin32
http://sourceforge.net/projects/pywin32/files/pywin32/Build%20218/pywin32-218.win32-py2.7.exe/download

- Python WMI
https://pypi.python.org/packages/any/W/WMI/WMI-1.4.9.win32.exe

- Debugging Tools for Windows
http://www.microsoft.com/whdc/devtools/debugging/default.mspx
  Set up symbols, if so desired.

- Microsoft !exploitable
  Copy the !exploitable dll (msec.dll) to winext directory.
  (probably C:\Program Files\Debugging Tools for Windows (x86)\winext)
http://msecdbg.codeplex.com/

- Add debugging tools (specifically cdb.exe) to your PATH.
  (probably C:\Program Files\Debugging Tools for Windows (x86)\)

- Copy the foe.yaml config file from configs\examples\ to a configs
  and modify as necessary.

- Copy seed files to the seedfiles directory