Sanitizers are tools for detecting certain classes of bugs in code. The operating principle varies, though sanitizers commonly (but not always) rely on some form of compile-time instrumentation to change code in ways that expose bugs at runtime. Fuchsia uses a variety of sanitizers to discover and diagnose dangerous bugs that are otherwise difficult to find.
Sanitizers are enabled by build-time flags. Sanitizer builds are continuously exercised on Fuchsia's Continuous Integration (CI) and Commit Queue (CQ), and serve Fuchsia C/C++ and Rust developers.
Developers typically benefit from sanitizers with no special action required, and only need to pay attention to sanitizers when they detect a bug. However certain limitations apply. Continue reading to learn what sanitizers are supported and how to use them.
Fuchsia currently supports the following sanitizers:
The following sanitizer behavior is available in the Zircon kernel:
pmm_checker) detects use-after-free bugs and stray DMAs.The following C/C++ compilation options are added by default to detect or prevent bugs at runtime:
-ftrivial-auto-var-init=pattern (see RFC) initializes automatic variables to a non-zero pattern to expose bugs related to reads from uninitialized memory.Lastly, Fuchsia uses libFuzzer{:.external} and syzkaller{:external} to perform coverage-directed fuzz testing. Fuzzers are similar to sanitizers in that they attempt to expose bugs in the code at runtime, and they are usually used in conjunction. Fuzzers are different from sanitizers in that fuzzers attempt to force the execution of production code into paths that may expose a bug.
Sanitizers are currently supported in local builds and in CI/CQ under the following configurations:
bringup.x64bringup.arm64core.x64core.arm64zbi_tests.x64zbi_tests.arm64In addition, sanitizers apply to host tools.
Tests for all of the above are exercised on CI/CQ on qemu{:.external} and Intel NUC.
Additional tryjobs for configurations defined under //vendor may be shown in Gerrit and in CI consoles for certain signed-in users. Look for configurations with -asan in their name.
The sanitizers listed above are applied to C/C++ code. In addition, LSan is applied to Rust code for detecting Rust memory leaks{:.external}.
To reproduce a sanitizer build, specify the sanitizer variants:
fx set {{ '<var label="product">product</var>' }} --variant asan-ubsan --variant host_asan-ubsan
Alternatively you may select to only instrument certain binaries:
fx set {{ '<var label="product">product</var>' }} --variant asan-ubsan/{{ '<var>executable_name</var>' }}
Specifically to detect use-after-free bugs in kernel code you will need to enable the kernel PMM checker.
Test as you normally would, in your local workflow or on CQ. If a sanitizer detects an issue during test runtime then the test failure will include a descriptive error message indicating the nature of the problem and a stack trace pointing to the root cause.
Issues detected by sanitizers typically have similar root causes. You may be able to find references for prior work by searching Fuchsia bugs for a bug with some of the same keywords that you're seeing in the sanitizer output.
See also: UBSan issues on Open Projects.
Sanitizers expose bugs at runtime. Unless your code runs, such as within a test or generally on Fuchsia in such a way that‘s exercised by CI/CQ, sanitizers won’t be able to expose bugs in your code.
The best way to ensure sanitizer coverage for your code is to ensure test coverage under the same configuration. Consult the guide on test coverage.
Sanitizers may be suppressed for certain build targets. Most commonly this is used for issues that predate the introduction of sanitizer support, especially for issues in third party code that the Fuchsia project doesn't own.
Suppressed sanitizers should be considered tech debt, as they not only hide old bugs but keep you from discovering new bugs as they're introduced to your code. Ideally new suppressions should not be added, and existing suppressions should be removed and the underlying bugs fixed.
Suppressing sanitizers may be done by editing the BUILD.gn file that defines your executable target as follows:
executable("please_fix_the_bugs") { ... # TODO(fxbug.dev/12345): delete the below and fix the memory bug. deps += [ "//build/config/sanitizers:suppress-asan-stack-use-after-return" ] # TODO(fxbug.dev/12345): delete the below and fix the memory bug. deps += [ "//build/config/sanitizers:suppress-asan-container-overflow" ] # TODO(fxbug.dev/12345): delete the below and fix the memory leak. deps += [ "//build/config/sanitizers:suppress-lsan.DO-NOT-USE-THIS" ] # TODO(fxbug.dev/12345): delete the below and fix undefined behavior. configs += [ "//build/config:temporarily_disable_ubsan_do_not_use" ] }
The example above demonstrates suppressing all sanitizers. However you should at most suppress sanitizers that are causing failures. Please track suppressions by filing a bug and referencing it in the comment as shown above.
Sanitizer errors may be flaky if the code under test's behavior is non-deterministic. For instance a memory leak may only happen under certain race conditions. If sanitizer errors appear flaky, consult the guide on testing for flakiness in CQ.
When encountering a sanitizer issue, file a bug containing all the troubleshooting information that's available.
Example: Issue 73214: ASAN use-after-scope in blobfs
The bug report contains:
Ongoing work:
Areas for future work:
unsafe {} code blocks or across FFI{:.external} calls, or detecting undefined behavior bugs.