docs/development/testing/fuzzing/fidl-fuzzing.md - fuchsia - Git at Google

 # Fuzzing FIDL

 Note: This is an area of active development. The instructions and results below should be considered
 experimental.

 ## Fuzzing FIDL servers with libFuzzer on Fuchsia

 Fuchsia includes experimental support for writing FIDL fuzzers in the style of FIDL unit tests.

 ### Quick-start guide

 An example fuzzer is defined in [`//examples/fuzzers/fidl`][example]. If you
 are not familiar with fuzzers, see the [overview](overview.md). Fuzzing FIDL servers with libFuzzer
 on Fuchsia requires GN targets that will generate a fuzz target and writing some code to provide an
 instance of the server to be fuzzed.

 1. Add `fuzzers = {protocol = "fully.qualified.fidl.ProtocolName"}` to your `fidl()` GN target.
 1. Depending on the language you are using:
     * At the C++ level (easy mode):
       Use the `FIDL_FUZZER_DEFINITION()` macro in `//sdk/lib/fidl/cpp/fuzzing/server_provider.h` to
       define a server provider for your interface and server implementation class. This will
       automatically define the C symbols described below. See
       `//examples/fuzzers/fidl` for a reference example.
     * At the C level (hard mode):
       Implement a library that defines the following symbols:
       * `zx_status_t fuzzer_init()`: Instantiate server implementation.
       * `zx_status_t fuzzer_connect(zx_handle_t, async_dispatcher_t*)`: Bind server implementation
          to the channel handle. Optionally, use the dispatcher if your server can be fuzzed on the
          same thread as fuzzer clients (see [note on threading](#a-note-about-threading)).
       * `zx_status_t fuzzer_disconnect(zx_handle_t, async_dispatcher_t*)`: Unbind server
          implementation from the channel handle.
       * `zx_status_t fuzzer_clean_up()`: Clean up server implementation.
       If any of these returns a status other than `ZX_OK`, then the fuzzer will cleanup and halt.
 1. Define a `fidl_protocol_fuzzer()` GN target. Specify:
     * `fidl = //path/to:fidl_gn_target` (the `fidl()` target mentioned above).
     * `protocol = "fully.qualified.fidl.ProtocolName"`.
     * `deps = [... :your_library ...]` (the one mentioned above, defining `fuzzer_...` symbols).
     * Anything else needed for a [`fuzzer`](build-a-fuzzer.md#fuzzer) GN target that fuzzes your
       server.
 1. Add `fidl_protocol_fuzzer()` target to `fuzzers = [ ... ]` in a new or existing
    [`fuzzers_package`](build-a-fuzzer.md#fuzzers_package) GN target.

 ### Implementation details

 The bulk of a FIDL server implementation fuzz target is C++ code generated by `fidlgen` that expects
 a handful of C symbols to provide an API to the FIDL server implementation. The generated code
 contains a global `async::Loop`, bound to its initial thread, that is reused for the client side of
 a FIDL connection on each run of the fuzz target. `LibFuzzer` repeatedly invokes the same fuzz
 target with different inputs. The generated fuzz target code will:

 * Invoke `fuzzer_init()`, initializing the server to be fuzzed.
 * Instantiates a pair of `zx::channel`s.
 * Initializes an `fidl::InterfacePtr` of the appropriate type, bound to a channel end and the loop's
   dispatcher.
 * Invoke `fuzzer_connect(raw_server_channel_handle, loop->dispatcher())`, establishing a connection
   with the server and allowing the server to opt in to using the same dispatcher as the client if
   its API is compatible with such a scheme (see [note on threading](#a-note-about-threading)).
 * Invoke a method through its `fidl::InterfacePtr`.
 * Set its `async::Loop` to run-until-idle.
 * Synchronize with the method's callback via a `zx::event`.
 * Invoke `fuzzer_disconnect(raw_server_channel_handle, loop->dispatcher())`, allowing the server to
   clean up its end of the connection.
 * Invoke `fuzzer_clean_up()` to tear down the server instance.

 #### Allocation of fuzz target input to FIDL messages

 In broad strokes, the first two bytes are used to select a protocol and method pair from among those
 defined in the FIDL source file. In the case of a FIDL file that contains many protocols, but only
 one is enabled in the fuzzer, discovery of meaningful inputs relies on `LibFuzzer`'s coverage-guided
 engine to deduce that the first bytes of a certain form cause the fuzzer to exercise almost no code.

 After a protocol and method pair are identified, the remaining bytes are carved up as follows:

 * Each type has a trait that defines the minimum number of bytes it requires.
 * If insufficient bytes are input, the fuzz target exits immediately.
 * Otherwise the "slack" bytes that exceed the minimum required for the method parameters are divided
 evenly among parameters and an allocation trait for each type is invoked to construct an object of
 the appropriate type using at most `MinForParam + SlackForParam` bytes.

 In terms of allocation trait details, collection and numeric types have relatively natural
 interpretations based on the collection of bytes to be transformed into an object. Handles are
 treated like numeric types, which can be expected to lead to errors when the server attempts to
 exercise them.

 #### A note about threading
 {#a-note-about-threading}

 It is **highly** desirable to keep the fuzz target single-threaded. That is, use
 `ServerProviderDispatcherMode::kFromCaller` in C++ or use the `async_dispatcher_t*` passed to
 `fuzzer_connect` in C. This is preferred because it increases the likelihood that bugs found by the
 fuzzer will be consistently reproducible.

 ## Fuzzing the FIDL host tools with AFL

 Additionally, past experimental efforts have fuzzed the FIDL compiler using
 [afl-fuzz](http://lcamtuf.coredump.cx/afl/).

 ### Build afl-fuzz

 Download and build it, then `export AFL_PATH` to be whatever path you downloaded and built it with.

 ### Patch the parser to not trap on invalid syntax

 afl-fuzz treats crashes as interesting but the parser currently calls `__builtin_trap()`
 when it encounters invalid syntax. Remove that line in
 [parser.h](/zircon/tools/fidl/include/fidl/parser.h) - it's in the `Parser::Fail()` method.

 ### Build the `fidl` tool with afl-fuzz's instrumentation

 Clear any existing build and then build with the afl-fuzz compiler wrappers.

 ```
 cd $ZIRCON_DIR
 rm -fr build-x86
 PATH=$PWD/prebuilt/downloads/clang+llvm-x86_64-linux/bin/:$PATH:$AFL_PATH make \
   build-x86/tools/fidl HOST_TOOLCHAIN_PREFIX=afl-
 ```

 Adjust if you're not building on x86 Linux, etc.

 ### Run the fuzzer

 The parser includes some examples to use as inputs.
 As FIDL becomes adopted we can expand our inputs to include all of the different protocols
 declared across our tree, but for now we use what's in `tools/fidl/examples`.

 ```
 $AFL_PATH/afl-fuzz -i tools/fidl/examples -o fidl-fuzz-out build-x86/tools/fidl dump '@@'
 ```

 ### Results

 Running against the source from early May 2017, there were no crashes or hangs after two days
 of fuzzing on a fairly fast machine. It ran over 300 million executions.

 [example]: /examples/fuzzers/fidl
	# Fuzzing FIDL

	Note: This is an area of active development. The instructions and results below should be considered
	experimental.

	## Fuzzing FIDL servers with libFuzzer on Fuchsia

	Fuchsia includes experimental support for writing FIDL fuzzers in the style of FIDL unit tests.

	### Quick-start guide

	An example fuzzer is defined in [`//examples/fuzzers/fidl`][example]. If you
	are not familiar with fuzzers, see the [overview](overview.md). Fuzzing FIDL servers with libFuzzer
	on Fuchsia requires GN targets that will generate a fuzz target and writing some code to provide an
	instance of the server to be fuzzed.

	1. Add `fuzzers = {protocol = "fully.qualified.fidl.ProtocolName"}` to your `fidl()` GN target.
	1. Depending on the language you are using:
	* At the C++ level (easy mode):
	Use the `FIDL_FUZZER_DEFINITION()` macro in `//sdk/lib/fidl/cpp/fuzzing/server_provider.h` to
	define a server provider for your interface and server implementation class. This will
	automatically define the C symbols described below. See
	`//examples/fuzzers/fidl` for a reference example.
	* At the C level (hard mode):
	Implement a library that defines the following symbols:
	* `zx_status_t fuzzer_init()`: Instantiate server implementation.
	* `zx_status_t fuzzer_connect(zx_handle_t, async_dispatcher_t*)`: Bind server implementation
	to the channel handle. Optionally, use the dispatcher if your server can be fuzzed on the
	same thread as fuzzer clients (see [note on threading](#a-note-about-threading)).
	* `zx_status_t fuzzer_disconnect(zx_handle_t, async_dispatcher_t*)`: Unbind server
	implementation from the channel handle.
	* `zx_status_t fuzzer_clean_up()`: Clean up server implementation.
	If any of these returns a status other than `ZX_OK`, then the fuzzer will cleanup and halt.
	1. Define a `fidl_protocol_fuzzer()` GN target. Specify:
	* `fidl = //path/to:fidl_gn_target` (the `fidl()` target mentioned above).
	* `protocol = "fully.qualified.fidl.ProtocolName"`.
	* `deps = [... :your_library ...]` (the one mentioned above, defining `fuzzer_...` symbols).
	* Anything else needed for a [`fuzzer`](build-a-fuzzer.md#fuzzer) GN target that fuzzes your
	server.
	1. Add `fidl_protocol_fuzzer()` target to `fuzzers = [ ... ]` in a new or existing
	[`fuzzers_package`](build-a-fuzzer.md#fuzzers_package) GN target.

	### Implementation details

	The bulk of a FIDL server implementation fuzz target is C++ code generated by `fidlgen` that expects
	a handful of C symbols to provide an API to the FIDL server implementation. The generated code
	contains a global `async::Loop`, bound to its initial thread, that is reused for the client side of
	a FIDL connection on each run of the fuzz target. `LibFuzzer` repeatedly invokes the same fuzz
	target with different inputs. The generated fuzz target code will:

	* Invoke `fuzzer_init()`, initializing the server to be fuzzed.
	* Instantiates a pair of `zx::channel`s.
	* Initializes an `fidl::InterfacePtr` of the appropriate type, bound to a channel end and the loop's
	dispatcher.
	* Invoke `fuzzer_connect(raw_server_channel_handle, loop->dispatcher())`, establishing a connection
	with the server and allowing the server to opt in to using the same dispatcher as the client if
	its API is compatible with such a scheme (see [note on threading](#a-note-about-threading)).
	* Invoke a method through its `fidl::InterfacePtr`.
	* Set its `async::Loop` to run-until-idle.
	* Synchronize with the method's callback via a `zx::event`.
	* Invoke `fuzzer_disconnect(raw_server_channel_handle, loop->dispatcher())`, allowing the server to
	clean up its end of the connection.
	* Invoke `fuzzer_clean_up()` to tear down the server instance.

	#### Allocation of fuzz target input to FIDL messages

	In broad strokes, the first two bytes are used to select a protocol and method pair from among those
	defined in the FIDL source file. In the case of a FIDL file that contains many protocols, but only
	one is enabled in the fuzzer, discovery of meaningful inputs relies on `LibFuzzer`'s coverage-guided
	engine to deduce that the first bytes of a certain form cause the fuzzer to exercise almost no code.

	After a protocol and method pair are identified, the remaining bytes are carved up as follows:

	* Each type has a trait that defines the minimum number of bytes it requires.
	* If insufficient bytes are input, the fuzz target exits immediately.
	* Otherwise the "slack" bytes that exceed the minimum required for the method parameters are divided
	evenly among parameters and an allocation trait for each type is invoked to construct an object of
	the appropriate type using at most `MinForParam + SlackForParam` bytes.

	In terms of allocation trait details, collection and numeric types have relatively natural
	interpretations based on the collection of bytes to be transformed into an object. Handles are
	treated like numeric types, which can be expected to lead to errors when the server attempts to
	exercise them.

	#### A note about threading
	{#a-note-about-threading}

	It is highly desirable to keep the fuzz target single-threaded. That is, use
	`ServerProviderDispatcherMode::kFromCaller` in C++ or use the `async_dispatcher_t*` passed to
	`fuzzer_connect` in C. This is preferred because it increases the likelihood that bugs found by the
	fuzzer will be consistently reproducible.

	## Fuzzing the FIDL host tools with AFL

	Additionally, past experimental efforts have fuzzed the FIDL compiler using
	[afl-fuzz](http://lcamtuf.coredump.cx/afl/).

	### Build afl-fuzz

	Download and build it, then `export AFL_PATH` to be whatever path you downloaded and built it with.

	### Patch the parser to not trap on invalid syntax

	afl-fuzz treats crashes as interesting but the parser currently calls `__builtin_trap()`
	when it encounters invalid syntax. Remove that line in
	[parser.h](/zircon/tools/fidl/include/fidl/parser.h) - it's in the `Parser::Fail()` method.

	### Build the `fidl` tool with afl-fuzz's instrumentation

	Clear any existing build and then build with the afl-fuzz compiler wrappers.

	```
	cd $ZIRCON_DIR
	rm -fr build-x86
	PATH=$PWD/prebuilt/downloads/clang+llvm-x86_64-linux/bin/:$PATH:$AFL_PATH make \
	build-x86/tools/fidl HOST_TOOLCHAIN_PREFIX=afl-
	```

	Adjust if you're not building on x86 Linux, etc.

	### Run the fuzzer

	The parser includes some examples to use as inputs.
	As FIDL becomes adopted we can expand our inputs to include all of the different protocols
	declared across our tree, but for now we use what's in `tools/fidl/examples`.

	```
	$AFL_PATH/afl-fuzz -i tools/fidl/examples -o fidl-fuzz-out build-x86/tools/fidl dump '@@'
	```

	### Results

	Running against the source from early May 2017, there were no crashes or hangs after two days
	of fuzzing on a fairly fast machine. It ran over 300 million executions.

	[example]: /examples/fuzzers/fidl