The build system is tightly coupled with how the continuous integration infrastructure (from here on referred to as just “infra”) discovers, aggregates, and runs tests. The high-level idea is that the test authors specify how they want their tests to run in GN and this information is propagated from the build system to the infra.
More specifically
The specification of a test's environments
in GN is what determines where and how a test is run. It is given as list of scopes of the following form:
environments = [ { dimensions = { <dimension key> = <value> ... } tags = ["<environment tags...>"] netboot = <boolean> }, ... ]
See guest_integration_tests for an example - and below for definitions of ‘dimension’ and ‘tags’
If no environments are specified for a test, than default behavior is as follows:
__is_fuchsia__
: test only runs in a QEMU instance running Fuchsia__is_linux__
: test only runs on a Linux machine__is_mac__
: test only runs on a Mac machine(1) means that hardware is opt-in; test authors must explicitly specify hardware environments in order to run tests there. The reasoning for this is that not all tests need to run on hardware, test authors know best whether that is the case, and that hardware is a scarce resource.
One may import //build/testing/environments.gni and use the environment-related convenience variables defined within. For example, basic_envs
includes all of the environments that are available to anyone without special consultation with the infra team.
dimensions
here refer to Swarming dimensions, where Swarming is the task distribution system used by the infra. A dimension is effectively a key-value pair that describes a bot property that can be targeted.
Tags are arbitrary strings that may be attached to an environment. Setting it amounts to removing the corresponding test from the normal testing pipeline; in order then for that test to run, infra support for a new builder (to run tests for particular tags) must be added. Labels are used for special tests that require different configurations. Before using tags, please consult with fuchsia-infra-team@google.com"
Netboot specifies whether to netboot instead of paving before running the tests in the shard for that environment. If omitted, it will be treated as false.
The test_plaforms
list in //build/testing/platforms.gni is the source of truth for what platforms are available for testing and what dimensions they possess to match against. Say an environment matches a platform entry if the former‘s dimensions
is a subscope of the latter; say an environment is valid for the current architecture if it matches a test_platforms
entry that doesn’t specify a cpu
value different than current_cpu
.
Environment validation happens at gn gen
-time and can be summed up as
Each environment must be valid for some architecture.
Each test must have an environment that is valid for the given architecture.
Suppose platforms.gni consisted of
test_platforms = [ { # P1 device_type = "QEMU" cpu = "x64" }, { # P2 device_type = "QEMU" cpu = "arm64" }, { # P3 device_type = "Intel NUC Kit NUC7i5DNHE" cpu = "x64" }, ]
and consider the specification of
environments = [ { # E1 dimensions = { device_type = "Intel NUC Kit NUC7i5DNHE" } }, { # E2 dimensions = { device_type = "QEMU" } }, ]
When current_cpu
is x64, E1 and E2 are both valid and match against P1 and P3 respectively: the test is scheduled to run on a NUC and in QEMU. When current_cpu
is arm64, E1 is invalid but is ignored as E2 is valid and matches P2: the test is scheduled to run in QEMU alone.