Handling results found through fuzzing

When your fuzzer runs, it searches for inputs that crash the program or violate checked conditions. When the fuzzer finds and reports such a test input, it is evidence of a bug that needs to be resolved.

Typically, you might use fx fuzz when first developing your fuzzer. This can often produce results immediately. After a fuzzer has been submitted it will be run at scale by ClusterFuzz, and any results it finds will be filed as bugs.

Handle results from fx fuzz

After running a fuzzer with fx fuzz, the tool can be used to report any artifacts found by running:

Additionally, output logs and any results are stored to the output directory. By default, this is:

A different location can be set passing the --output option to fx fuzz.

The most recent fuzzer run is symbolically linked to:

Crashes and other artifacts will have file name like {{ "<var>" }}type-of-finding{{ "</var>" }}.{{ "<var>" }}SHA1-hash-of-input{{ "</var>" }}. The file contents will be the input bytes themselves.

For example, examining a crash produced by a toy example might look like the following:

Reproduce a result

You can execute the fuzzer with this input again using fx fuzz repro. For example:

If the result is reproducible, this will produce a symbolized log including a stack trace. The top of this stack trace is likely to be the error handling by libFuzzer and/or the sanitizer, and the bottom will likely be the fuzzer engine itself.

For example, the relevant function where the panic occurred in the following stack trace is _toy_example_arbitrary_lib_rustc_static::toy_example:

For unreproducible results, you can still examine symbolized log from the original fuzzer execution for clues.

Attach a debugger

You may also want to attach a debugger when reproducing fuzzer results. By default, libFuzzer on Fuchsia creates a debug exception channel attached to the fuzzing thread in order to detect and handle crashes during fuzzing. Only one process may do this per thread, so debuggers are prevented from attaching.

To prevent libfuzzer from creating a debug exception channel, use the --debug option with fx fuzz.

For example, to use zxdb while reproducing a specific test case:

Now, in a separate terminal, start the fuzzer with your test case:

File fuzzing bugs

Note: The bug tracker is currently only open to Googlers.

It may be tempting to immediately fix the bug related to the fuzzer result, especially if the bug is obvious. No matter how trivial the bug is, please file a bug report!

To file a bug, please use the Fuzzing Bug template. This ensures you include certain labels, such as found-by-fuzzing, libfuzzer and Sec-TriageMe. This in turn helps the security team see where fuzzers are being used and stay aware of any critical issues they are finding.

Important: As with other potential security issues, bugs should be filed in the component of the code under test, and not in the Security component.

If you encounter problems or shortcomings in the fuzzing framework itself, open bugs or feature requests in the Security>libFuzzer component.

As with all potential security issues, you do not need to wait for triage to begin fixing the bug! Once fixed, reference the bug number in the commit message.

Handle bugs from ClusterFuzz

ClusterFuzz will file bugs automatically when it finds reproducible fuzzer results. If you are assigned such a bug, look for the following:

  • The Detailed Report will contain details about the result, including:

    • What type of result it is.
    • Whether it has security implications.
    • What revisions exhibited the behavior.
    • What stack frames appear to uniquely identify the crash.
  • The Reproducer Testcase will link to a fuzzer artifact. You can download this artifact and then use it to reproduce the fuzzer result as described above.

When you submit a fix so that a fuzzer stops producing an artifact from the input, ClusterFuzz will automatically close the bug.

Bugs found by fuzzing

Note: The bug tracker is currently only open to Googlers.

{% dynamic if user.is_googler %}

You can see bugs found in Fuchsia by fuzzing in Monorail.

You can also see graphs of this information using the Fuchsia fuzzing bug dashboard.

{% dynamic endif %}