docs/development/build/bazel_concepts/tests.md

# Bazel Tests

## Scope

This document describes how to define, build and run tests that exclusively
exist in the Bazel graph.

Fuchsia only supports Bazel **host tests** that meet the following requirements:

- These tests must be buildable exclusively with Bazel; i.e. they do not depend
  on artifacts generated by a prior Ninja invocation.

- These tests only include binaries and runtime files that can run on a
  compatible host system that is **NOT** connected to any Fuchsia device or
  emulator.

[fxbug.dev/546173288](https://fxbug.dev/546173288) tracks supporting target
tests in Bazel.

## Defining Fuchsia host tests in Bazel

Regular Bazel test rules (e.g. `rustc_test`, `go_test`) are not compatible with
Fuchsia test runners. They will not be visible to `fx test` and will not
be run by infra builders.

Hence, to define a Fuchsia-compatible host test in Bazel:

- Use language-specific host test wrapper from
  `//build/bazel/rules/host_tests:host_<xxx>_test.bzl`, such as `host_rustc_test()`,
  `host_go_test()`, etc.

  These are designed to be drop-in replacements for regular Bazel test rules,
  with extra arguments like `test_data`, `test_args`, so Fuchsia test runners
  can invoke them with the right runtime environment and arguments.

- Only if no language-specific wrappers are available for your test, use
  the generic `host_test()` wrapper from
  `//build/bazel/rules/host_tests:host_test.bzl`. For example:

  ```starlark
  load("//build/bazel/rules/host_tests:host_test.bzl", "host_test")
  load("@rules_sh//sh:sh_binary.bzl", "sh_binary")

  sh_binary(
      name = "test_bin",
      ...
  )

  host_test(
      name = "shell_test",
      binary = ":test_bin",
  )
  ```

  Note: **ALWAYS** prefer using language-specific wrappers if available, as they
  also implement extra Fuchsia-specific features, which are critical for parsing
  test results in infra.

- Do **NOT** assume the execution location of a test binary (e.g. from
  `out/default`), as it is launched from a directory whose exact path is not
  predictable.

  Bazel host tests **MUST** be hermetic and list all their runtime dependencies
  explicitly, just like GN host tests. Runtime dependencies can be provided
  through one of the mechanisms:

  - As `test_data` dependencies to a `host_test_data_files()` or
    `host_test_data_map()` target (see `//build/bazel/rules/host_tests:host_test_data.bzl`).
    These make the files available at runtime at fixed locations, relative to the
    test's execution directory, similar to the GN `host_test_data()` template.

  - As `data` attribute on a `host_xxx_test()` target.
    In this case, the files will be added to the test's runfiles, and can be
    accessed using a [runfiles library][bazel_runfiles].

  [bazel_runfiles]: /build/bazel/BAZEL_RUNFILES.md

- Use `test_suite()` instead of `filegroup()` to define groups of tests.
  Test suites can depend on tests and other test suites.

  Warning: Bazel skips tests from `test_suite`s if they are incompatible
  with the current configuration. See
  [Bazel test compatibility](#bazel-test-compatibility) for more details.

## Exporting Bazel host tests to `fx test` and infra builders

Defining the `host_xxx_test()` target alone will **NOT** make it visible to
`fx test` and, more importantly, to infra builders.

To make a Bazel host test visible to `fx test` and infra builders, write a
`bazel_test_suite` target in GN that lists the Bazel labels of one or more Bazel
host tests or suites, and add that target to a "tests" group in GN. For example:

`//src/foo/bar/BUILD.bazel`:

```bazel
load("//build/bazel/platforms:constraints.bzl", "HOST_OS_CONSTRAINTS")

test_suite(
  name = "host_tests",
  host_tests = [
    "//src/foo/bar/lib:lib_tests",
    "//src/foo/bar/util:util_tests",
  ]
  target_compatible_with = HOST_OS_CONSTRAINTS,
)
```

`//src/foo/bar/BUILD.gn`:

```gn
import("//build/bazel/bazel_test_suite.gni")

bazel_test_suite("bazel_tests") {
  host_tests = [
    # This references the target in BUILD.bazel.
    "//src/foo/bar:host_tests",
  ]
}

group("tests") {
  deps = [
    ":bazel_tests",
    ...
  ]
}
```

This will cause entries for the Bazel tests to appear in
`out/default/tests.json`. When in doubt, examine the content of this file after
`fx set` or `fx gen`. The `label` field for Bazel tests begins with `@`.

During development, it is possible to build and run test targets once they are
defined, before they are exported as described above. See the
[Running tests directly with Bazel](#running-tests-directly-with-bazel) section
below.

Note: Bazel tests do not show up in `tests.json` if the `export_bazel_tests`
GN argument is overridden to `false`.

### Best practices

It's recommended to avoid deep nesting of Bazel `test_suite` targets. Instead,
use just one `test_suite` Bazel target and one corresponding `bazel_test_suite`
GN target per source code sub-area (e.g. host tool or Fuchsia component).

## Running Fuchsia Bazel host tests

### Local testing

#### Running tests with `fx test`

To run Bazel host tests with `fx test`, they **MUST** be exported by following
the instructions in the
[Exporting Bazel host tests to `fx test` and infra builders](#exporting-bazel-host-tests-to-fx-test-and-infra-builders)
section above.

After the tests are exported, use `fx test --host <name_or_label>` to build and
run a Bazel host test locally, same as with GN-defined host tests.

The only difference is that the test will be built on demand by invoking Bazel
directly, skipping Ninja entirely.

Note that when using `fx test`:

- `<name_or_label>` will be subject to fuzzy matching, and `fx test` will report
  multiple candidates in case of ambiguity.

- `<name_or_label>` cannot reference a Bazel `test_suite()` label, just like it
  cannot reference a GN `group()` one.

See the [User guide for fx test](/docs/reference/testing/fx-test.md) for more
details about how to use `fx test`.

#### Running tests directly with `fx bazel test` {#running-tests-directly-with-bazel}

Caution: Only use this during local development, as it bypasses the mechanisms
that infra builders use to run tests.

You can use `fx bazel test --config=host <target_pattern>` to build and run
tests locally directly with Bazel. This does not require the extra plumbing to
make tests visible to `fx test` as described in the previous section, and has
the following advantages:

- `<target_pattern>` is a set of Bazel target patterns that can match "all test
  targets in a given package", in which case Bazel will skip non-test targets
  automatically.

- `<target_pattern>` can match `test_suite()` labels which can be convenient to
  run a set of host tests repeatedly during development.

- `bazel test` caches test results, and will only re-run the tests impacted by
  your latest changes since the last invocation.

  (Use `--nocache_test_results` flag to disable this).

- `<target_pattern>` can reference labels that are not visible in `tests.json`.
  This can be handy during development when adding new tests.

Warning: Bazel skips tests matched by `<target_pattern>` if they are
incompatible with the current configuration. See
[Bazel test compatibility](#bazel-test-compatibility) for more details.

Tip: If you have RBE enabled in your `args.gn`, use `fx bazel test
--config=remote --config=test <target_pattern>` to run your host tests on remote
test bots. This is noticeably slower but is a good way to verify that your
tests are truly hermetic before running them on infra.

### Infra testing

Only host tests that are listed in `tests.json` can be launched on infra test
bots. This applies equally to GN and Bazel host tests. So you **MUST** ensure
they are visible in `tests.json` by following the
[Exporting Bazel host tests to `fx test` and infra builders](#exporting-bazel-host-tests-to-fx-test-and-infra-builders)
section above, otherwise they will **NOT** be continuously tested on infra.

Infra builds Bazel tests using `fint`, which directly calls `bazel build` after
running `ninja`, to build all Bazel tests that are listed in `bazel_test_suite`
GN targets.

There is no explicit distinction between GN and Bazel-defined host tests when
running them on infra test runners, or when collecting results. In particular,
infra does not run tests using `bazel test`. Instead, it takes the binaries
produced by `bazel build` and runs them directly, just like it does with host
tests from ninja.

## Debug symbol propagation

Host test binaries compiled with debug symbols must have their unstripped ELF
binaries registered so that tooling can symbolize backtraces and correlate LLVM
code coverage profiles (`-profile-correlate=binary`).

For general background on how Fuchsia manages Bazel debug symbols, see
[Technical Note on Debug Symbol Generation](/build/bazel/debug_symbols/README.md).

### Authoring host test rules with debug symbols

The generic `host_test()` rule exposes the `unstripped_binary` field in its
`FuchsiaHostTestInfo` provider:

- If `unstripped_binary` is not explicitly set, `host_test()` inspects the
  underlying `binary` attribute:
  - If `binary` provides `DebugPackageInfo` (C++ targets), it uses
    `DebugPackageInfo.unstripped_file`.
  - If `binary` provides `CrateInfo` (Rust targets), it uses
    `CrateInfo.output`.

- If the test uses a wrapper script launcher (such as `rust_test_parser` in
  `host_rustc_test()`), the macro **MUST** explicitly forward the original
  unstripped binary target via `unstripped_binary`:

  ```starlark
  host_test(
      name = name,
      binary = wrapper_script,
      unstripped_binary = ":" + binary_name,
      ...
  )
  ```

### Propagation to infra and debuginfod

When Bazel host tests are exported to the GN build:

1. **Test Query**: `bazel_tests_utils.py` runs a `bazel cquery` using
   `//build/bazel/starlark/FuchsiaHostTestInfo.cquery`, extracting the
   execroot path of `unstripped_binary` for all registered host tests.

2. **Host Test Manifest**: The paths are normalized relative to the Ninja build
   directory and written to
   `${root_build_dir}/bazel_host_tests.debug_symbols.json`.

3. **GN Build API Integration**: `//:bazel_test_suites` in `BUILD.gn` attaches
   `debug_symbol_manifests` metadata pointing to
   `bazel_host_tests.debug_symbols.json`, which is consumed by the
   `build_api_module("debug_symbols")` target.

4. **Artifactory Upload**: In CI/CQ builds, Artifactory processes
   `debug_symbols.json` and uploads the unstripped ELF binaries to cloud
   storage / debuginfod servers, allowing tools like `covargs` to resolve
   Build IDs.

### Local development and `fx coverage`

When running coverage locally at desk, uncommitted binaries have newly generated
Build IDs that are not present on debuginfod servers.

To support local symbol correlation, `fx coverage` runs:

```bash
python3 ${FUCHSIA_DIR}/build/bazel/scripts/copy_bazel_debug_symbols.py ${FUCHSIA_BUILD_DIR}
```

This script reads `bazel_host_tests.debug_symbols.json`, computes the GNU Build
ID for each ELF binary, and copies or symlinks the unstripped binary into
`${root_build_dir}/.build-id/xx/yyyyyyyy.debug`. Tooling such as `ffx coverage`
and `llvm-profdata` then resolves these binaries via `.symbol-index.json`.

### FAQ

#### Bazel test compatibility

If any test in a `test_suite()` (or any of its dependencies) is incompatible
with the current configuration (e.g., using `target_compatible_with`), that test
will be skipped during evaluation of the suite instead of failing the build.

The same is true when using patterns such as `//...` and `:all` on the command line.

For more information, see the [handling of incompatible targets][bazel_target_compatible_with].

When running a `test_suite()` or using a target pattern, developers must verify that
the test(s) of interest are reported as `PASSED` (and not `SKIPPED`).

[bazel_target_compatible_with]: https://bazel.build/reference/be/common-definitions#common.target_compatible_with