Contributing Tests to CTF

This guide will walk you through the process of writing a CTF test for a FIDL API.

{% set fidl_library = “fuchsia.examples” %} {% set fidl_service = “fuchsia.examples.Echo” %}

To test a FIDL API, we write a test that uses the API‘s client bindings in the SDK to interact with the FIDL service. For this guide we’ll be testing an example FIDL service {{ fidl_service }} from the library {{ fidl_library }}. Throughout this guide, you can replace the library and service with your own values to match your use case.

The test will consist of two components: The first is a test driver binary which implements the core test logic, and is the part of your test that will be released in CTF. The second is a test realm which provides the capabilities and dependencies that we want to test, is always built from the sources at HEAD, and is not released as part of CTF.

Requirements

Tests must be written in C, C++, or Rust.
Tests must only depend on APIs, ABIs, and tools in partner facing SDKs.

Note: Exceptions for dependencies are made by updating the allowlist here.

Note: The examples in this guide use synchronous FIDL clients, but this is not a requirement. Test authors can choose to use synchronous or asynchronous clients.

If you are writing a CTF test for a FIDL service whose fidl target is not in the partner SDK category, please see this section.

Concepts

Steps

1. Setup

First create a directory for the test. The directory name should match the name of the FIDL library. You can copy these commands to generate some scaffolding:

{% set test_root = “sdk/ctf/tests/fidl/” + fidl_library %} {% set test_component_name = fidl_service|lower + “_test” %}

{C/C++}

mkdir {{ test_root }}
mkdir {{ test_root }}/meta/
touch {{ test_root }}/meta/{{ test_component_name }}.cml
touch {{ test_root }}/BUILD.gn
touch {{ test_root }}/main.cc

{Rust}

mkdir {{ test_root }}
mkdir {{ test_root }}/meta/
# Rust tests need an additional component to offer the subpackaged test runner.
touch {{ test_root }}/meta/{{ test_component_name }}_root.cml
touch {{ test_root }}/meta/{{ test_component_name }}.cml
touch {{ test_root }}/BUILD.gn
touch {{ test_root }}/main.rs

2. Create the test realm

The test realm is a component whose sole purpose is to expose the FIDL API that we want to test. The test realm component is always built from the HEAD of the current Fuchsia branch. This is what makes the CTF test a compatibility test: The test and the FIDL capability it's testing are built at different versions.

For convenience, the test realm component can be defined anywhere in the source tree but we prefer if all realms are defined in //sdk/ctf/test_realm/BUILD.gn.

To create the realm, add contents like the following to //sdk/ctf/test_realm/BUILD.gn:

{% includecode gerrit_repo="fuchsia/fuchsia" gerrit_path="sdk/ctf/test_realm/BUILD.gn" region_tag="example" adjust_indentation="auto" %}

{% set test_realm_package = fidl_library + “_test_realm” %}

Then create the test realm's component manifest at //sdk/ctf/test_realm/meta/{{ test_realm_package }}.cml:

{% includecode gerrit_repo="fuchsia/fuchsia" gerrit_path="sdk/ctf/test_realm/meta/fuchsia.example_test_realm.cml" region_tag="example" adjust_indentation="auto" %}

Finally, add the realm‘s label to the list in //sdk/ctf/build/ctf_test_realms.gni. This will cause the build to include the test realm as a subpackage of your test. We’ll explain this in more detail, later.

3. Write the BUILD.gn file

Add contents like the following to //{{ test_root }}/BUILD.gn to define an executable, test driver component, and package for your test. Be sure to add the :tests target as a dependency of //sdk/ctf/tests/fidl:tests in order to include the test in the build graph for CI and CQ builds.

{C/C++}

{% includecode gerrit_repo="fuchsia/fuchsia" gerrit_path="sdk/ctf/tests/examples/fidl/fuchsia.examples/cc/BUILD.gn" region_tag="build" adjust_indentation="auto" %}

{Rust}

{% includecode gerrit_repo="fuchsia/fuchsia" gerrit_path="sdk/ctf/tests/examples/fidl/fuchsia.examples/rust/BUILD.gn" region_tag="build" adjust_indentation="auto" %}

4. Implement the test driver

This component implements the core logic for your test.

First we need to create the component manifest. Add contents like the following to {{ test_component_name }}.cml:

{C/C++}

{% includecode gerrit_repo="fuchsia/fuchsia" gerrit_path="sdk/ctf/tests/examples/fidl/fuchsia.examples/cc/meta/fuchsia.examples.echo_test.cml" region_tag="example" adjust_indentation="auto" %}

{Rust}

This test root component includes //sdk/ctf/meta/rust.shard.cml, which defines the rust test runner as a subpackage. The echo_test component must be started in the subpackaged-runner-env.

// echo_test_root.cml

{% includecode gerrit_repo="fuchsia/fuchsia" gerrit_path="sdk/ctf/tests/examples/fidl/fuchsia.examples/rust/meta/fuchsia.examples.echo_test_root.cml" region_tag="example" adjust_indentation="auto" %}

The test component. The test realm is offered from the test root rather than the test itself.

// echo_test.cml

{% includecode gerrit_repo="fuchsia/fuchsia" gerrit_path="sdk/ctf/tests/examples/fidl/fuchsia.examples/rust/meta/fuchsia.examples.echo_test.cml" region_tag="example" adjust_indentation="auto" %}

The package URL {{ test_realm_package }}#meta/default.cm loads the test realm we created earlier from a subpackage. Every time this test runs it receives a new version of this subpackage which is built from the current commit, regardless of whether the test itself is built from the current commit or obtained as a prebuilt.

Next we need to implement the executable. Add contents like the following to the test's source file:

{C/C++}

{% includecode gerrit_repo="fuchsia/fuchsia" gerrit_path="sdk/ctf/tests/examples/fidl/fuchsia.examples/cc/main.cc" region_tag="example" adjust_indentation="auto" %}

{Rust}

{% includecode gerrit_repo="fuchsia/fuchsia" gerrit_path="sdk/ctf/tests/examples/fidl/fuchsia.examples/rust/main.rs" region_tag="example" adjust_indentation="auto" %}

5. Running the test

These instructions require you to open several terminal tabs. Follow the insructions for each tab, from left to right:

{Build Fuchsia}

fx set core.x64 --with //{{ test_root }}:tests
fx build

{Run the emulator}
```
ffx emu start --headless
```
{Serve packages}
```
fx serve
```
{Stream logs}
```
ffx log
```

{Run the tests}

fx test -v {{ test_component_name }}
# -v enables verbose output.

If you need additional help debugging at this step, please reach out to fuchsia-ctf-team@google.com.

6. Submit the changes

If the tests pass, send your changes for review. After submission the tests willji automatically be included in the next CTF release when the next milestone branch is cut.

Testing experimental FIDL APIs

You can follow this guide to write a CTF test for a FIDL API that is not in the partner SDK category, but you must not release the test in CTF until the API has been added to the partner category. To prevent the test from being released, set release_in_ctf = false on the test package:

{% includecode gerrit_repo="fuchsia/fuchsia" gerrit_path="sdk/ctf/tests/examples/experimental_fidl/BUILD.gn" region_tag="norelease_example" adjust_indentation="auto" %}

Contributing Tests to CTF

Requirements

Concepts

Steps

1. Setup

2. Create the test realm

3. Write the BUILD.gn file

4. Implement the test driver

5. Running the test

6. Submit the changes

Testing experimental FIDL APIs

See Also