To promote test reuse, Fuchsia test suite components should avoid using RealmBuilder directly. Instead, tests should use FIDL to call a separate test realm factory component (TRF) which creates the test realm and returns a RealmProxy connection. The RealmProxy connection provides access to the realm's exposed capabilities.
This pattern makes it easier to reuse a test in CTF because the test suite and and test realm factory are separated by FIDL interfaces, enabling different versions of the test suite to run against the same test realm factory and vice versa. Tests that follow this pattern will also be reusable for different types of testing in the future.
See the list of examples below for additional help.
Rather than directly using RealmBuilder, tests should use FIDL to call a test realm factory component which creates the test realm. The realm factory component returns a RealmProxy FIDL connection which the test suite can use to connect to the realm's exposed capabilities. By convention, RealmFactory FIDL protocols accept a test specific RealmOptions
FIDL table as an argument to support configuration.
The test suite and test realm factory must belong to separate packages. This allows releasing the test suite's package in CTF without also releasing the test realm factory. At build time, subpackaging combines both components into a single package.
If a RealmFactory FIDL protocol will be used by CTF tests, it should have FIDL availability annotations. The protocol must also be added to the partner_internal
SDK category to indicate that the API can be used by CTF tests and must maintain backward compatibility but is not be exposed to SDK users or the partner
category if it should be available to SDK users.
A puppet is a test-only component that runs in the test realm and exposes a FIDL protocol to the test suite. The test suite connects to the protocol using the RealmProxy connection, and the puppet interacts with the other components in the realm by following commands sent from the test suite.
Puppets allow a test to interact with the test realm without exposing the realm's implementation details. For example, a puppet could be used to provide test input data to a component or to inspect its state.
Puppets can be written to support a single test or as general purpose components that can be reused in different test realms.
For an example of a puppet component, see the Archivist test puppet which emits logs and Inspect data to Archivist, and is controlled by this FIDL protocol. Any test may use the Archivist test puppet to inject arbitrary logs and Inspect data into test realms.
Note: If the puppet's FIDL protocol will be used by CTF tests, remember to add the protocol to the partner_internal
SDK category to indicate that the API can be used to CTF tests and must maintain backward compatibility, but is not be exposed to SDK users.
Some tests provide input data to the component under test such as structured configuration values or data read from a directory. When using a test realm factory component, there are at least two ways to provide input data to your component:
RealmFactory/CreateRealm
using some field of a RealmOptions
FIDL table.The first approach is the simplest and is usually required when the test data must be provided before the component starts running, but contributors should take care to avoid a leaky abstraction with this approach. For example: if the component reads input data from a /config/data
directory capability, consider passing the data from the test suite (as a VMO) and have the test realm serve this directory, rather than passing a handle to a fuchsia.io.Directory
capability.
RealmBuilder makes it possible to define ‘local components’ to stub FIDL protocols or to record FIDL traffic. However, with TRF the test suite and the test realm run in different processes. Instead, consider creating a puppet component that the test suite can use to send stub requests or responses to the component under test.
Below are some examples of test suites and test realm factory components.
If your test uses RealmBuilder to run and test a component, it should wrap all RealmBuilder code in a test realm factory component instead. Even if you do not plan to run your test in CTF, you must adopt this pattern to take advantage of the Fuchsia team's future plans to make tests more reusable in different contexts, such as performance testing, system (non-hermetic) testing, and end-to-end testing.
See this section on mocking FIDL interactions.
The RealmProxy protocol does not provide access to non-protocol capabilities. Consider creating a puppet to mediate access to non-protocol capabilities.