| # The Fuchsia Test Runner Framework | |
| The Fuchsia [Component Framework][cf] allows developers to create components in | |
| a variety of languages and runtimes. Fuchsia's own code uses a diverse mix of | |
| programming languages for components, including C/C++, Rust, Dart, and Go. | |
| The Test Runner Framework uses Component Framework [runners][runners] as an | |
| integration layer between various testing runtimes and a common Fuchsia protocol | |
| for launching tests and receiving their results. This makes for an inclusive | |
| design that on one hand allows developers to bring their language and testing | |
| framework of choice, and on the other hand allows building and testing Fuchsia | |
| on a variety of systems and targeting different hardware. | |
| ## The Test Manager | |
| The `test_manager` component is responsible for running tests on a Fuchsia | |
| device. Test manager exposes the | |
| [`fuchsia.test.manager.RunBuilder`][fidl-test-manager] protocol, which allows | |
| launching test suites. | |
| Each test suite is launched as a child of test manager. Test suites are offered | |
| capabilities by test manager that enable them to do their work while | |
| maintaining isolation between the test and the rest of the system. For instance | |
| hermetic tests are given the capability to log messages, but are not given the | |
| capability to interact with real system resources outside of their sandbox. | |
| Test manager uses only one capability from the test realm, a controller protocol | |
| that test suites expose. This is done to ensure hermeticity (test results aren't | |
| affected by anything outside of their intended sandbox) and isolation (tests | |
| don't affect each other or the rest of the system). | |
| The test manager controller itself is offered to other components in the system | |
| in order to integrate test execution with various developer tools. Tests can | |
| then be launched with such tools as [`fx test`][fx-test] and [`ffx`][ffx]. | |
| ## The test suite protocol {#test-suite-protocol} | |
| The test suite protocol, [`fuchsia.test.Suite`][fidl-test-suite], is used by the | |
| test manager to control tests, such as to invoke test cases and to collect their | |
| results. | |
| Test authors typically don't need to implement this protocol. Instead, they rely | |
| on a [test runner](#test-runners) to do this for them. For instance, you might | |
| write a test in C++ using the GoogleTest framework, and then use | |
| [`gtest_runner`](#gtest-runner) in your [component manifest][component-manifest] | |
| to integrate with the Test Runner Framework. | |
| ## Test runners {#test-runners} | |
| ### A language and runtime-inclusive framework | |
| Test runners are reusable adapters between the Test Runner Framework and common | |
| languages & frameworks used by developers to write tests. They implement the | |
| [`fuchsia.test.Suite`][fidl-test-suite] protocol on behalf of the test author, | |
| allowing developers to write idiomatic tests for their language and framework of | |
| choice. | |
| Component manifests for simple unit tests can be [generated][unit-tests] | |
| by the build rules. Generated component manifests for v2 tests will include the | |
| appropriate test runner based on their build definition. For instance a test | |
| executable that depends on the GoogleTest library will include the | |
| [GoogleTest runner](#gtest-runner) in its generated manifest. | |
| ### Inventory of test runners | |
| The following test runners are currently available for general use: | |
| #### GoogleTest runner {#gtest-runner} | |
| A runner for tests written in C/C++ using the GoogleTest framework. | |
| Use this for all tests written using GoogleTest. | |
| Common GoogleTest features are supported, such as disabling tests, running | |
| only specified tests, running the same test multiple times, etc'. | |
| Standard output, standard error, and logs are captured from the test. | |
| In order to use this runner, add the following to your component manifest: | |
| ```json5 | |
| { | |
| include: [ "//src/sys/test_runners/gtest/default.shard.cml" ] | |
| } | |
| ``` | |
| By default GoogleTest test cases run serially (one test case at a time). | |
| #### GoogleTest (Gunit) runner {#gunit-runner} | |
| A runner for tests written in C/C++ using the GUnit framework. | |
| Use this for all tests written using the gUnit flavor of GoogleTest. | |
| Note: Gtest and Gunit testing framework differ in flag names, so we have a | |
| separate runner for gunit. | |
| Common GoogleTest features are supported, such as disabling tests, running | |
| only specified tests, running the same test multiple times, etc'. | |
| Standard output, standard error, and logs are captured from the test. | |
| In order to use this runner, add the following to your component manifest: | |
| ```json5 | |
| { | |
| include: [ "sys/testing/gunit_runner.shard.cml" ] | |
| } | |
| ``` | |
| By default test cases run serially (one test case at a time). | |
| #### Rust runner {#rust-runner} | |
| A runner for tests written in the Rust programming language and following Rust | |
| testing idioms. | |
| Use this for all idiomatic Rust tests (i.e. tests with modules that set the | |
| attribute `[cfg(test)]`). | |
| Common Rust testing features are supported, such as disabling tests, running | |
| only specified tests, running the same test multiple times, etc'. | |
| Standard output, standard error, and logs are captured from the test. | |
| In order to use this runner, add the following to your component manifest: | |
| ```json5 | |
| { | |
| include: [ "//src/sys/test_runners/rust/default.shard.cml" ] | |
| } | |
| ``` | |
| By default Rust test cases run in parallel, at most 10 cases at a time. | |
| #### ELF test runner {#elf-test-runner} | |
| The simplest test runner - it waits for your program to terminate, then reports | |
| that the test passed if the program returned zero or that it failed for any | |
| non-zero return value. | |
| Use this test runner if your test is implemented as an ELF program (for instance | |
| an executable written in C/C++) but it does not use a common testing framework | |
| that's supported by existing runners and you'd rather not implement a bespoke | |
| test runner. | |
| In order to use this runner, add the following to your component manifest: | |
| ```json5 | |
| { | |
| include: [ "sys/testing/elf_test_runner.shard.cml" ] | |
| } | |
| ``` | |
| If you are [using in-tree unit test GN templates][component-unit-tests], | |
| and you are not already using a test framework with a dedicated test runner, | |
| add the following to your build deps: | |
| ``` | |
| fuchsia_unittest_package("my-test-packkage") { | |
| // ... | |
| deps = [ | |
| // ... | |
| "//src/sys/testing/elftest", | |
| ] | |
| } | |
| ``` | |
| Note: If you see the error message "Component has a \`program\` block defined, | |
| but doesn't specify a \`runner\`" for your test, this indicates you are not using a | |
| test framework with a dedicated test runner, and you should add the above dependency. | |
| ### Controlling parallel execution of test cases | |
| When using `fx test` to launch tests, they may run each test case in sequence or | |
| run multiple test cases in parallel up to a given limit. The default | |
| parallelism behavior is determined by the test runner. To manually control the | |
| number of test cases to run in parallel use test spec: | |
| ```gn | |
| fuchsia_test_package("my-test-pkg") { | |
| test_components = [ ":my_test_component" ] | |
| test_specs = { | |
| # control the parallelism | |
| parallel = 10 | |
| } | |
| } | |
| ``` | |
| ### Running test multiple times | |
| To run a test multiple times use: | |
| ```posix-terminal | |
| fx test --count=<n> <test_url> | |
| ``` | |
| If an iteration times out, no further iteration will be executed. | |
| ### Passing arguments | |
| Custom arguments to the tests can be passed using `fx test`: | |
| ```posix-terminal | |
| fx test <test_url> -- <custom_args> | |
| ``` | |
| Individual test runners have restrictions on these custom flags: | |
| #### GoogleTest runner {#gtest-runner-custom-arg} | |
| Note the following known behavior changes: | |
| **--gtest_break_on_failure** - Instead use: | |
| ```posix-terminal | |
| fx test --break-on-failure <test_url> | |
| ``` | |
| The following flags are restricted and the test fails if any are passed as | |
| fuchsia.test.Suite provides equivalent functionality that replaces them. | |
| - **--gtest_filter** - Instead use: | |
| ```posix-terminal | |
| fx test --test-filter=<glob_pattern> <test_url> | |
| ``` | |
| `--test-filter` may be specified multiple times. Tests that match any of the | |
| given glob patterns will be executed. | |
| - **--gtest_also_run_disabled_tests** - Instead use: | |
| ```posix-terminal | |
| fx test --also-run-disabled-tests <test_url> | |
| ``` | |
| - **--gtest_repeat** - See [Running test multiple times](#running_test_multiple_times). | |
| - **--gtest_output** - Emitting gtest json output is not supported. | |
| - **--gtest_list_tests** - Listing test cases is not supported. | |
| #### GoogleTest (Gunit) runner {#gunit-runner-custom-arg} | |
| Note the following known behavior changes: | |
| **--gunit_break_on_failure** - Instead use: | |
| ```posix-terminal | |
| fx test --break-on-failure <test_url> | |
| ``` | |
| The following flags are restricted and the test fails if any are passed as | |
| fuchsia.test.Suite provides equivalent functionality that replaces them. | |
| - **--gunit_filter** - Instead use: | |
| ```posix-terminal | |
| fx test --test-filter=<glob_pattern> <test_url> | |
| ``` | |
| `--test-filter` may be specified multiple times. Tests that match any of the | |
| given glob patterns will be executed. | |
| - **--gunit_also_run_disabled_tests** - Instead use: | |
| ```posix-terminal | |
| fx test --also-run-disabled-tests <test_url> | |
| ``` | |
| - **--gunit_repeat** - See [Running test multiple times](#running_test_multiple_times). | |
| - **--gunit_output** - Emitting gtest json/xml output is not supported. | |
| - **--gunit_list_tests** - Listing test cases is not supported. | |
| #### Rust runner {#rust-runner-custom-arg} | |
| The following flags are restricted and the test fails if any are passed as | |
| fuchsia.test.Suite provides equivalent functionality that replaces them. | |
| - **\<test_name_matcher\>** - Instead use: | |
| ```posix-terminal | |
| fx test --test-filter=<glob_pattern> <test_url> | |
| ``` | |
| `--test-filter` may be specified multiple times. Tests that match any of the | |
| given glob patterns will be executed. | |
| - **--nocapture** - Output is printed by default. | |
| - **--list** - Listing test cases is not supported. | |
| ### A runtime-agnostic, runtime-inclusive testing framework {#inclusive} | |
| Fuchsia aims to be inclusive, for instance in the sense that | |
| developers can create components (and their tests) in their language and runtime | |
| of choice. The Test Runner Framework itself is language-agnostic by design, with | |
| individual test runners specializing in particular programming languages or test | |
| runtimes and therefore being language-inclusive. Anyone can create and use new | |
| test runners. | |
| Creating new test runners is relatively easy, with the possibility of sharing | |
| code between different runners. For instance, the GoogleTest runner and the Rust | |
| runner share code related to launching an ELF binary, but differ in code for | |
| passing command line arguments to the test and parsing the test's results. | |
| ## Temporary storage | |
| To use temporary storage in your test, add the following to your component manifest: | |
| ```json5 | |
| { | |
| include: [ "//src/sys/test_runners/tmp_storage.shard.cml" ] | |
| } | |
| ``` | |
| At runtime, your test will have read/write access to `/tmp`. | |
| The contents of this directory will be empty when the test starts, and will be | |
| deleted after the test finishes. | |
| [Tests that don't specify a custom manifest][component-unit-tests] and instead | |
| rely on the build system to generate their component manifest can add the | |
| following dependency: | |
| ```gn | |
| fuchsia_unittest_package("foo-tests") { | |
| deps = [ | |
| ":foo_test", | |
| "//src/sys/test_runners:tmp_storage", | |
| ] | |
| } | |
| ``` | |
| ## Exporting custom files {#custom-artifacts} | |
| To export custom files from your test, use the *custom_artifacts* storage | |
| capability. The contents of *custom_artifacts* are copied out at the conclusion | |
| of a test. | |
| To use *custom_artifacts* in your test, add the following to your component | |
| manifest: | |
| ```json5 | |
| { | |
| use: [ | |
| { | |
| storage: "custom_artifacts", | |
| rights: [ "rw*" ], | |
| path: "/custom_artifacts", | |
| }, | |
| ], | |
| } | |
| ``` | |
| At runtime, your test will have read/write access to `/custom_artifacts`. | |
| The contents of this directory will be empty when the test starts, and will be | |
| deleted after the test finishes. | |
| See the [custom artifact test example][custom-artifact-example]. To run it, add | |
| `//examples/tests/rust:tests` to your build, then run: | |
| ```posix-terminal | |
| fx test --ffx-output-directory <output-dir> custom_artifact_user | |
| ``` | |
| After the test concludes, `<output-dir>` will contain the `artifact.txt` file | |
| produced by the test. | |
| ## Hermeticity | |
| In the context of software testing, Hermeticity refers to the isolation of a | |
| test or test suite from external factors and dependencies, ensuring that it | |
| produces consistent and reliable results regardless of changes in the | |
| surrounding environment. A hermetic test is self-contained and doesn't rely on | |
| external systems or data that might change unexpectedly, leading to flaky or | |
| non-deterministic test outcomes. | |
| *Hermeticity does not mean protection from in-stable platform or routed | |
| capabilities/APIs. If an API surface is in-stable for certain components in the | |
| system, then they will be instable for tests and dependent components and will | |
| help catch regressions due to any direct or in direct changes to system's API | |
| surface.* | |
| Ability to write fully, provably hermetic tests is Fuchsia's **Testing | |
| Superpower**. There are two type of test hermeticity: | |
| - Capability: The Test does not [use][manifests-use] or [offer][manifests-offer] | |
| any capabilities from the [test root's](#test-roles) parent. These tests don't | |
| have access to any system capabilities which can affect larger system. Due to | |
| this property of hermetic tests they can be run in parallel and will not flake | |
| due to cross-talk or shared state, it improves stability and performance of | |
| tests. | |
| - Package: The Test does not [resolve][resolvers] any components outside of the | |
| test package. A hermetically packaged test does not have any implicit contract | |
| with platform packages. This provides a way to update them without affecting | |
| system packages and avoids dependence on incompatible packaged dependencies. | |
| Hermeticity does not apply to platform APIs/capabilities which are available to | |
| all components in the system. For eg | |
| - clock | |
| - Kernel provided identifiers, such as koids. | |
| - Framework capabilities provided to all components which allow clients to | |
| mutate component manager state, and while this is not strictly hermetic it is | |
| component manager's responsibility to ensure isolation | |
| The tests should be careful when using these APIs/capabilities. | |
| The tests are by default hermetic unless explicitly stated otherwise. | |
| ### Hermetic capabilities for tests | |
| There are some capabilities which all tests can use which do not violate test | |
| hermeticity: | |
| | Protocol | Description | | |
| | -----------| ------------| | |
| | `fuchsia.boot.WriteOnlyLog` | Write to kernel log | | |
| | `fuchsia.logger.LogSink` | Write to syslog | | |
| | `fuchsia.process.Launcher` | Launch a child process from the test package | | |
| | `fuchsia.diagnostics.ArchiveAccessor` | Read diagnostics output by components in the test | | |
| The hermeticity is retained because these capabilities are carefully curated | |
| to not allow tests to affect the behavior of system components outside the test | |
| realm or of other tests. | |
| To use these capabilities, there should be a use declaration added to test's | |
| manifest file: | |
| ```json5 | |
| // my_test.cml | |
| { | |
| use: [ | |
| ... | |
| { | |
| protocol: [ | |
| "{{ '<var label="protocol">fuchsia.logger.LogSink</var>' }}" | |
| ], | |
| }, | |
| ], | |
| } | |
| ``` | |
| Tests are also provided with some default storage capabilities which are | |
| destroyed after the test finishes execution. | |
| | Storage Capability | Description | Path | | |
| | ------------------ | ----------- | ---- | | |
| | `data` | Isolated data storage directory | `/data` | | |
| | `cache` | Isolated cache storage directory | `/cache` | | |
| | `tmp` | Isolated in-memory [temporary storage directory](#temporary_storage) | `/tmp` | | |
| Add a use declaration in test's manifest file to use these capabilities. | |
| ```json5 | |
| // my_test.cml | |
| { | |
| use: [ | |
| ... | |
| { | |
| storage: "{{ '<var label="storage">data</var>' }}", | |
| path: "{{ '<var label="storage path">/data</var>' }}", | |
| }, | |
| ], | |
| } | |
| ``` | |
| The framework also provides some [capabilities][framework-capabilities] to all | |
| the components and can be used by test components if required. | |
| ### Hermetic component resolution {#hermetic-resolver} | |
| Hermetic test components are launched in a realm that utilizes the hermetic | |
| component resolver. This resolver disallows resolving URLs outside of the | |
| test's package. This is necessary for enforcing hermeticity, as we don't | |
| want the availability of an arbitrary component on the system or in an | |
| associated package server to affect the outcome of a test. | |
| Attempts to resolve a component not in the test's package will be met with a | |
| `PackageNotFound` error and the following message in the syslog: | |
| ``` | |
| failed to resolve component fuchsia-pkg://fuchsia.com/[package_name]#meta/[component_name]: package [package_name] is not in the set of allowed packages... | |
| ``` | |
| You can avoid this error by including any components your test relies on | |
| to the test package - see [this CL](https://fxrev.dev/608222) for an example of | |
| how to do this, or by using [subpackages]: | |
| ```gn | |
| # BUILD.gn | |
| import("//build/components.gni") | |
| fuchsia_test_package("simple_test") { | |
| test_components = [ ":simple_test_component" ] | |
| subpackages = [ "//path/to/subpackage:subpackage" ] | |
| } | |
| ``` | |
| ```json5 | |
| // test.cml | |
| { | |
| ... | |
| children: [ | |
| { | |
| name: "child", | |
| url: "subpackage#meta/subpackaged_component.cm", | |
| }, | |
| ], | |
| ... | |
| } | |
| ``` | |
| See [this CL](https://fxrev.dev/784304) as an example of using subpackages. | |
| Note: Subpackages are not available in the sdk yet. If you are not able to | |
| include a dependent component in your test package you can add below option | |
| to your test manifest file to selectively allow resolution of some packages: | |
| ```json5 | |
| // my_component_test.cml | |
| { | |
| ... | |
| facets: { | |
| "fuchsia.test": { | |
| "deprecated-allowed-packages": [ "non_hermetic_package" ], | |
| }, | |
| }, | |
| ... | |
| } | |
| ``` | |
| ### Non-hermetic tests | |
| These tests can access some pre-defined capabilities outside of the test realm. | |
| A capability accessed by non-hermetic test from outside its test realm is called | |
| a *system capability*. | |
| To use a system capability, a test must explicitly mark itself to run in | |
| non-hermetic realm as shown below. | |
| ```gn | |
| # BUILD.gn (in-tree build rule) | |
| fuchsia_test_component("my_test_component") { | |
| component_name = "my_test" | |
| manifest = "meta/my_test.cml" | |
| deps = [ ":my_test_bin" ] | |
| # This runs the test in "system-tests" non-hermetic realm. | |
| test_type = "system" | |
| } | |
| ``` | |
| After integrating with the build rule, the test can be executed as | |
| ``` | |
| fx test <my_test> | |
| ``` | |
| Or for out-of-tree developers | |
| ``` | |
| ffx test run --realm <realm_moniker> <test_url> | |
| ``` | |
| where `realm_moniker` should be replaced with `/core/testing/system-tests` for | |
| above example. | |
| Possible values of `test_type`: | |
| | Value | Description | | |
| | ----- | ----------- | | |
| | `chromium` | [Chromium test realm] | | |
| | `ctf` | [CTF test realm] | | |
| | `device` | [Device tests] | | |
| | `drm` | [DRM tests] | | |
| | `starnix` | [Starnix tests] | | |
| | `system_validation` | [System Validation Tests] | | |
| | `system` | [Legacy non hermetic realm][system-test-realm] with access to some system capabilities. | | |
| | `test_arch` | [Test Architecture Tests] | | |
| | `vfs-compliance` | [VFS compliance tests] | | |
| | `vulkan` | [Vulkan tests] | | |
| [Learn][create-test-realm] how to create your own test realm. | |
| ### Non-hermetic legacy test realms | |
| These are legacy test realms created before we had | |
| [Test Manager as a Service][test-manager-as-a-service]. We are in process of | |
| porting these realms. If your tests depend on one of these realms, it should | |
| explicitly mark itself to run in the legacy realm as shown below. | |
| ```json5 | |
| // my_component_test.cml | |
| { | |
| include: [ | |
| // Select the appropriate test runner shard here: | |
| // rust, gtest, etc. | |
| "//src/sys/test_runners/rust/default.shard.cml", | |
| // This includes the facet which marks the test type as 'starnix'. | |
| {{ '<strong>' }}"//src/devices/testing/starnix_test.shard.cml",{{ '</strong>' }} | |
| ], | |
| program: { | |
| binary: "bin/my_component_test", | |
| }, | |
| {{ '<strong>' }} | |
| use: [ | |
| { | |
| protocol: [ "fuchsia.vulkan.loader.Loader" ], | |
| }, | |
| ],{{ '</strong>' }} | |
| } | |
| ``` | |
| The shard includes following facet in the manifest file: | |
| ```json5 | |
| {% includecode gerrit_repo="fuchsia/fuchsia" gerrit_path="src/starnix/tests/starnix_test.shard.cml" %} | |
| ``` | |
| Possible values of `fuchsia.test.type`: | |
| | Value | Description | | |
| | ----- | ----------- | | |
| | `hermetic` | Hermetic realm | | |
| | `chromium-system` | Chromium system test realm | | |
| | `google` | Google test realm | | |
| ## Restricted logs | |
| By default, a test will fail if it logs a message with a severity of | |
| `ERROR` or higher. See this [guide][restricted-logs] for more information. | |
| ## Performance | |
| When writing a test runner that launches processes, the runner needs to | |
| provide a [library loader][loader-service] implementation. | |
| Test runners typically launch individual test cases in separate processes to | |
| achieve a greater degree of isolation between test cases. However this can come | |
| at a significant performance cost. To mitigate this, the test runners listed | |
| above use a [caching loader service][caching-loader-service] which reduces the | |
| extra overhead per process launched. | |
| ## Test roles {#test-roles} | |
| Components in the test realm may play various roles in the test, as follows: | |
| - Test root: The component at the top of a test's component tree. The URL for | |
| the test identifies this component, and the test manager will invoke the | |
| [`fuchsia.test.Suite`][test-suite-protocol] exposed by this component to | |
| drive the test. | |
| - Test driver: The component that actually runs the test, and implements | |
| (either directly or through a [test runner](#test-runners) the | |
| [`fuchsia.test.Suite`][test-suite-protocol] protocol. Note that the test | |
| driver and test root may be, but are not necessarily, the same component: | |
| the test driver could be a subcomponent of the test root which re-exposes its | |
| `fuchsia.test.Suite`, for example. | |
| - Capability provider: A component that provides a capability that the test | |
| will exercise somehow. The component may either provide a "fake" | |
| implementation of the capability for test, or a "real" implementation that | |
| is equivalent to what production uses. | |
| - Component under test: A component that exercises some behavior to be tested. | |
| This may be identical to a component from production, or a component written | |
| specifically for the test intended to model production behavior. | |
| ## Troubleshooting {#troubleshooting} | |
| This section contains common issues you may encounter while developing test components | |
| with the Test Runner Framework. If one of your test components fails to run, you may see | |
| an error like the following from `fx test`: | |
| ```none {:.devsite-disable-click-to-copy} | |
| Test suite encountered error trying to run tests: getting test cases | |
| Caused by: | |
| The test protocol was closed. This may mean `fuchsia.test.Suite` was not configured correctly. | |
| ``` | |
| To address the issue, explore the following options: | |
| - [The test is using wrong test runner](#troubleshoot-wrong-test-runner) | |
| - [The test failed to expose `fuchsia.test.Suite` to test manager](#troubleshoot-test-root) | |
| - [The test driver failed to expose `fuchsia.test.Suite` to the root](#troubleshoot-test-routing) | |
| - [The test driver does not use a test runner](#troubleshoot-test-runner) | |
| ### The test is using wrong test runner {#troubleshoot-wrong-test-runner} | |
| If you encountered this error during test enumeration then probably you are | |
| using the wrong test runner. | |
| For example: your Rust test file might be running the | |
| test without using Rust test framework (i.e it is a simple Rust binary with its | |
| own main function). In this case change your test manifest file to use | |
| [elf_test_runner](#elf-test-runner). | |
| Read more about in-built [test runners](#test-runners). | |
| ### The test failed to expose `fuchsia.test.Suite` to test manager {#troubleshoot-test-root} | |
| This happens when the test root fails to expose `fuchsia.test.Suite` from the | |
| [test root](#test-roles). The simple fix is to add an `expose` declaration: | |
| ```json5 | |
| // test_root.cml | |
| expose: [ | |
| ... | |
| { | |
| protocol: "fuchsia.test.Suite", | |
| from: "self", // If a child component is the test driver, put `from: "#driver"` | |
| }, | |
| ], | |
| ``` | |
| ### The test driver failed to expose `fuchsia.test.Suite` to the root {#troubleshoot-test-routing} | |
| Your test may fail with an error similar to the following if the `fuchsia.test.Suite` | |
| protocol is not properly exposed: | |
| ```none {:.devsite-disable-click-to-copy} | |
| ERROR: Failed to route protocol `/svc/fuchsia.test.Suite` from component | |
| `/test_manager/...`: An `expose from #driver` declaration was found at `/test_manager/...` | |
| for `/svc/fuchsia.test.Suite`, but no matching `expose` declaration was found in the child | |
| ``` | |
| If the [test driver and test root](#test-roles) are different components, the test driver | |
| must also expose `fuchsia.test.Suite` to its parent, the test root. | |
| To address this issue, ensure the [test driver](#test-roles) component manifest includes | |
| the following `expose` declaration: | |
| ```json5 | |
| // test_driver.cml | |
| expose: [ | |
| ... | |
| { | |
| protocol: "fuchsia.test.Suite", | |
| from: "self", | |
| }, | |
| ], | |
| ``` | |
| ### The test driver does not use a test runner {#troubleshoot-test-runner} | |
| The [test driver](#test-roles) must use the appropriate [test runner](#test-runners) | |
| corresponding to the language and test framework the test is written with. | |
| For example, the driver of a Rust test needs the following declaration: | |
| ```json5 | |
| // test_driver.cml | |
| include: [ "//src/sys/test_runners/rust/default.shard.cml" ] | |
| ``` | |
| Also, if the test driver is a child of the [test root](#test-roles), you need | |
| to offer it to the driver: | |
| ```json5 | |
| // test_root.cml | |
| offer: [ | |
| { | |
| runner: "rust_test_runner", | |
| to: [ "#driver" ], | |
| }, | |
| ], | |
| ``` | |
| ## Further reading | |
| - [Complex topologies and integration testing][integration-testing]: testing | |
| interactions between multiple components in isolation from the rest of the | |
| system. | |
| [caching-loader-service]: /src/sys/test_runners/src/elf/elf_component.rs | |
| [cf]: /docs/concepts/components/v2/ | |
| [Chromium test realm]: /src/sys/testing/meta/chromium_test_realm.shard.cml | |
| [component-manifest]: /docs/concepts/components/v2/component_manifests.md | |
| [component-unit-tests]: /docs/development/components/build.md#unit-tests | |
| [create-test-realm]: /docs/development/testing/components/create_test_realm.md | |
| [CTF test realm]: /docs/development/testing/ctf/test_collection.md | |
| [custom-artifact-example]: /examples/tests/rust/custom_artifact_test.rs | |
| [Device tests]: /src/devices/testing/devices_test_realm.shard.cml | |
| [DRM tests]: /src/media/testing/drm_test_realm.shard.cml | |
| [ffx]: /docs/development/tools/ffx/overview.md | |
| [fidl-test-manager]: /sdk/fidl/fuchsia.test.manager/test_manager.fidl | |
| [fidl-test-suite]: /sdk/fidl/fuchsia.test/suite.fidl | |
| [framework-capabilities]: /docs/concepts/components/v2/capabilities/protocol.md#framework | |
| [fx-test]: https://fuchsia.dev/reference/tools/fx/cmd/test | |
| [integration-testing]: /docs/development/testing/components/integration_testing.md | |
| [loader-service]: /docs/concepts/process/program_loading.md#the_loader_service | |
| [manifests-offer]: https://fuchsia.dev/reference/cml#offer | |
| [manifests-use]: https://fuchsia.dev/reference/cml#use | |
| [resolvers]: /docs/concepts/components/v2/capabilities/resolver.md | |
| [restricted-logs]: /docs/development/diagnostics/test_and_logs.md#restricting_log_severity | |
| [runners]: /docs/concepts/components/v2/capabilities/runner.md | |
| [Starnix tests]: /src/sys/testing/meta/starnix-tests.shard.cml | |
| [subpackages]: /docs/concepts/components/v2/subpackaging.md | |
| [System Validation Tests]: /src/testing/system-validation/meta/system_validation_test_realm.shard.cml | |
| [system-test-realm]: /src/sys/testing/meta/system-tests.shard.cml | |
| [Test Architecture Tests]: /src/sys/testing/meta/test-arch-tests.shard.cml | |
| [test-manager-as-a-service]: /docs/contribute/governance/rfcs/0202_test_manager_as_a_service.md | |
| [test-suite-protocol]: /docs/concepts/components/v2/realms.md | |
| [unit-tests]: /docs/development/components/build.md#unit_tests_with_generated_manifests | |
| [VFS compliance tests]: /src/sys/component_manager/tests/capability_provider_vfs_compliance/vfs_compliance_test_realm.shard.cml | |
| [Vulkan tests]: /src/lib/vulkan/vulkan_test_realm.shard.cml |