<<../components/_v2_banner.md>>
The Fuchsia Component Framework allows developers to create components in a variety of languages and runtimes. Fuchsia's own code uses a diverse mix of programming languages for components, including C/C++, Rust, Dart, and Go.
The Test Runner Framework uses Component Framework runners as an integration layer between various testing runtimes and a common Fuchsia protocol for launching tests and receiving their results.
The test suite protocol, fuchsia.test.Suite, is used by the test manager to control tests, such as to invoke test cases and to collect their results.
Test authors typically don't need to implement this protocol. Instead, they rely on a test runner to do this for them. For instance, you might write a test in C++ using the GoogleTest framework, and then use gtest_runner in your component manifest to integrate with the Test Runner Framework.
Test runners are reusable adapters between the Test Runner Framework and common languages & frameworks used by developers to write tests. They implement the fuchsia.test.Suite protocol on behalf of the test author, allowing developers to write idiomatic tests for their language and framework of choice.
Component manifests for simple unit tests can be [generated][unit-tests] by the build rules. Generated component manifests for v2 tests will include the appropriate test runner based on their build definition. For instance a test executable that depends on the GoogleTest library will include the GoogleTest runner in its generated manifest.
The following test runners are currently available for general use:
A runner for tests written in C/C++ using the GoogleTest framework. Use this for all tests written using GoogleTest.
Common GoogleTest features are supported, such as disabling tests, running only specified tests, running the same test multiple times, etc'. Standard output, standard error, and logs are captured from the test.
In order to use this runner, add the following to your component manifest:
{ include: [ "src/sys/test_runners/gtest/default.shard.cml" ] }
By default GoogleTest test cases run serially (one test case at a time).
A runner for tests written in the Rust programming language and following Rust testing idioms. Use this for all idiomatic Rust tests (i.e. tests with modules that set the attribute [cfg(test)]).
Common Rust testing features are supported, such as disabling tests, running only specified tests, running the same test multiple times, etc'. Standard output, standard error, and logs are captured from the test.
In order to use this runner, add the following to your component manifest:
{ include: [ "src/sys/test_runners/rust/default.shard.cml" ] }
By default Rust test cases run in parallel, at most 10 cases at a time.
A runner for tests written in the Go programming language and following Go testing idioms. Use this for all tests written in Go using import "testing".
Common Go testing features are supported, such as disabling tests, running only specified tests, running the same test multiple times, etc'. Standard output, standard error, and logs are captured from the test.
In order to use this runner, add the following to your component manifest:
{ include: [ "src/sys/test_runners/gotests/default.shard.cml" ] }
By default Go test cases run in parallel, at most 10 cases at a time.
The simplest test runner - it waits for your program to terminate, then reports that the test passed if the program returned zero or that it failed for any non-zero return value.
Use this test runner if your test is implemented as an ELF program (for instance an executable written in C/C++) but it does not use a common testing framework that‘s supported by existing runners and you’d rather not implement a bespoke test runner.
In order to use this runner, add the following to your component manifest:
{ include: [ "src/sys/test_runners/elf/default.shard.cml" ] }
There is no notion of parallelism since tests that use this runner don't have a notion of multiple test cases.
When using fx test to launch tests, they may run each test case in sequence or run multiple test cases in parallel up to a given limit. The default parallelism behavior is determined by the test runner.
To manually set the parallelism level for test cases, run the following:
fx shell run-test-suite --parallel=5 <test_url>
To use temporary storage in your test, add the following to your component manifest:
{ include: [ "src/sys/test_runners/tmp_storage.shard.cml" ] }
At runtime, your test will have read/write access to /tmp. The contents of this directory will be empty when the test starts, and will be deleted after the test finishes.
A test is hermetic if it uses or offers no capabilities from the test root's parent. As a rule of thumb, tests should be hermetic, but sometimes a test requires a capability that cannot be injected in the test realm.
In the context of hermetic tests, a capability that originates from outside of the test's realm is called a system capability.
Components in the test realm may play various roles in the test, as follows:
fuchsia.test.Suite protocol. This role may be, but is not necessarily, owned by the test root.