This document introduces the Fuchsia Testing Framework (FTF) along with fundamental concepts and terminology around testing in Fuchsia.
Note: The Fuchsia Testing Framework is only supported from Components v2. All component-related concepts discussed herein refer to Components v2 unless otherwise noted.
Like any other Fuchsia program, a Fuchsia test is a component. Like all components, a Fuchsia test component may be viewed as a realm, or tree of components, also called a test realm. The component at the root of the test realm is known as the test root.
For a component to be a test, it must manifests-expose and implement the fuchsia.test.Suite
protocol.
Components in the test realm may play various roles, as defined in test roles.
The test suite protocol, fuchsia.test.Suite
, is the protocol implemented by a test that the test manager uses to run tests and obtain information about them. Most commonly, the test code itself (i.e., the test driver) is written using a language-specific test library, and the protocol is implemented by a test runner on the test's behalf.
The fuchsia.test.Suite protocol is the way for Components v2 tests to enumerate their test cases and report results. This differs from Components v1, where test results are reported through stdout and exit codes. Most commonly, a test will not implement the fuchsia.test.Suite
protocol from scratch, but would instead use a test runner that integrates a language-native testing library with the protocol. The the does this by declaring it uses the appropriate test runner in the [test driver][test-roles]‘s manifest. For example, for a C++ gtest, the test driver’s manifest would contain the following:
// test_driver.cml { ... "use": [ ... // Use the gtest runner { "runner": "gtest_runner", }, ], "expose": [ ... // Test driver must still expose fuchsia.test.Suite { "protocol": "/svc/fuchsia.test.Suite", "from": "self", }, ], }
However, it's cumbersome for a test to manually implement this protocol. For this reason, the Fuchsia Testing Frameworks provides a set of special runners called test runners. A test runner integrates a test driver that uses a language-native test library (such as C++ gtest
or rust libtest
) with the test suite protocol. If your component declares it uses the appropriate test runner in its component manifest, you can write your test natively against the language-specific test library and do not need to manually export results under the test suite protocol.
A test is hermetic if it uses or offers no capabilities from the test root's parent. As a rule of thumb, tests should be hermetic, but sometimes a test requires a capability that cannot be injected in the test realm.
In the context of hermetic tests, a capability that originates from outside of the test's realm is called a system capability.
Components in the test realm may play various roles in the test, as follows:
fuchsia.test.Suite
protocol. This role may be, but is not necessarily, owned by the test root.