This document standardizes the way we define and organize FIDL tests in the GN build system, respecting the following goals:
fx test fidl_rust_conformance_tests
, then Go should use fx test fidl_go_conformance_tests
. There should be no need to continually look these up in the contributing doc because they're all different.fx test
.fx test
, etc.This document uses the following terminology:
fx gn help toolchain
General guidelines:
_tests
rather than the singular _test
.The last point means preferring full names like fidl_rust_conformance_tests
over contextual names like conformance_tests
. It might seem verbose and redundant to repeat “fidl” and “rust” at the directory, package, component, and binary level. But the fact is that these names must be unique, and it is better to make them unique in a consistent way rather than remembering odd rules like fidl-bindings-test
is for Dart and fidl-test
is for C.
Names should use the following scheme, joining parts with hyphens or underscores:
tool [ bindings ] [ category [ subcategory ] ] tests
Where tool is one of:
And the other parts are:
Every BUILD.gn file that defines tests should include a "tests"
group:
group("tests") { testonly = true deps = [ ... ] # not public_deps }
These are aggregated in “tests” groups of BUILD.gn files in parent directories. The root “tests” group (for some portion of the codebase, e.g. src/lib/fidl/BUILD.gn) should be included in bundles/fidl/BUILD.gn. This enables fx set ... --with //bundles/fidl:tests
to include all FIDL tests in the build. (The tests are also run in CQ because //bundles/buildbot:core
includes //bundles/fidl:tests
.)
Assume we have a :fidl_foo_tests_bin
target that produces a fidl_foo_tests
binary. To wrap this in a package, start with a unittest_package
:
import("//build/test/test_package.gni") import("//build/testing/environments.gni") unittest_package("fidl-foo-tests") { deps = [ ":fidl_foo_tests_bin" ] tests = [ { name = "fidl_foo_tests" environments = basic_envs }, ] }
We can now run the test by package name (fx test fidl-foo-tests
) or by component name (fx test fidl_foo_tests
). For single-test packages like this, use the component name in documentation (e.g. contributing_to_fidl.md, "Test:"
lines in commit messages).
For multiple device tests, collect them all in a single package instead of making separate packages. For example, suppose we split fidl_foo_tests
into fidl_foo_unit_tests
and fidl_foo_integration_tests
:
import("//build/test/test_package.gni") import("//build/testing/environments.gni") unittest_package("fidl-foo-tests") { deps = [ ":fidl_foo_unit_tests_bin", ":fidl_foo_integration_tests_bin", ] tests = [ { name = "fidl_foo_unit_tests" environments = basic_envs }, { name = "fidl_foo_integration_tests" environments = basic_envs }, ] }
Most of the time unittest_package
is enough. If your test needs any component features, services, etc., you must instead use test_package
and write a component manifest file:
# BUILD.gn import("//build/test/test_package.gni") import("//build/testing/environments.gni") test_package("fidl-foo-tests") { deps = [ ":fidl_foo_tests_bin" ] tests = [ { name = "fidl_foo_tests" environments = basic_envs }, ] } # meta/fidl_foo_tests.cmx { "program": { "binary": "test/fidl_foo_tests" }, "sandbox": { "services": [ "fuchsia.logger.LogSink", # some example services "fuchsia.process.Launcher" ] } }
The manifest path defaults to meta/fidl_too_tests.cmx in this case. As with unittest_package
, you can include multiple test components. Each one will need its own component manifest file.
Assume we have a :fidl_bar_tests
target that produces a fidl_bar_tests
host test binary, defined using one of the test templates: test
, go_test
, etc. We must ensure that GN is in $host_toolchain
when it reaches that target, otherwise it will try to build it for Fuchsia:
groups("tests") { testonly = true deps = [ ":fidl_bar_tests($host_toolchain)" ] }
(Always put ($host_toolchain)
in the BUILD.gn file's tests
group, not in //bundles/fidl:tests.)
This will create a test_spec entry named host_x64/fidl_bar_tests
, which will end up in out/default/tests.json:
{ "command": [ "host_x64/fidl_bar_tests", "--test.timeout", "5m" ], "cpu": "x64", "label": "//PATH/TO/BAR:fidl_bar_tests(//build/toolchain:host_x64)", "name": "host_x64/fidl_bar_tests", "os": "linux", "path": "host_x64/fidl_bar_tests", "runtime_deps": "host_x64/gen/PATH/TO/BAR/fidl_bar_tests.deps.json" }
Running fx test fidl_bar_tests
works because of the “name” field in tests.json.
Tests that run both on host and device fall in two categories. In the first category, the test target simply builds under either toolchain. For example:
import("//build/test/test_package.gni") import("//build/testing/environments.gni") rustc_test("fidl_rust_conformance_tests") { # host test name ... } unittest_package("fidl-rust-tests") { deps = [ ":fidl_rust_conformance_tests" ] tests = [ { name = "fidl_rust_conformance_tests" # device test name environments = basic_envs }, ] } group("tests") { testonly = true deps = [ ":fidl_rust_conformance_tests($host_toolchain)", ":fidl-rust-tests", ] }
We can now run the test both ways:
fx test fidl_rust_conformance_tests --device
fx test fidl_rust_conformance_tests --host
In the second category, the device and host tests share source code, but they are sufficiently different that they must be defined by separate targets. This time, use the same name for the host test target and the component name. This will require choosing a different name for the device test target. For example:
import("//build/test/test_package.gni") import("//build/testing/environments.gni") source_set("conformance_test_sources") { ... } test("fidl_hlcpp_conformance_tests") { # host test name ... deps = [ ":conformance_test_sources", ... ] } test("fidl_hlcpp_conformance_tests_fuchsia") { # We had to append _fuchsia to distinguish this from the host test target. # But we want the binary name to be fidl_hlcpp_conformance_tests to match # up with the same-named test component in the unittest_package. output_name = "fidl_hlcpp_conformance_tests" ... deps = [ ":conformance_test_sources", ... ] } unittest_package("fidl-hlcpp-tests") { deps = [ ":fidl_hlcpp_conformance_tests_fuchsia" ] tests = [ { name = "fidl_hlcpp_conformance_tests" # device test name environments = basic_envs }, ] } group("tests") { testonly = true deps = [ ":fidl_hlcpp_conformance_tests($host_toolchain)", ":fidl-hlcpp-tests" ] }
Now, we can run the test both ways:
fx test fidl_hlcpp_conformance_tests --device
fx test fidl_hlcpp_conformance_tests --host
C++ tests in Zircon are usually defined using zx_test
. This template behaves differently from most others like test
, rustc_test
, go_test
, etc.: it appends -test
to the binary name by default. Do not rely on this, for two reasons:
_tests
suffix, not -test
.Instead, specify the target and binary name explicitly:
zx_test("fidlc_unit_tests") { output_name = "fidlc_unit_tests" # explicit binary name ... }
Rust libraries are often defined like this:
rustc_library("baz") { with_unit_tests = true ... }
This automatically creates a baz_test
target that builds a baz_lib_test
binary. Do not use this, for two reasons:
_tests
suffix, not -test
.Instead of with_unit_tests
, write a separate rustc_test
target with an appropriate name:
rustc_library("baz") { ... } rustc_test("fidl_baz_tests") { ... }
Suppose we have the following test structure:
We should have test targets for the leaves:
fx test fidl_rust_conformance_tests
fx test fidl_rust_integration_tests
We should not make additional targets for running various subsets of the tests. Using fx test
, we can already
fx test //path/to/fidl/rust
fx test //path/to/fidl/rust --device
fx test //path/to/fidl/rust --host