The Fuchsia Platform Surface Area (PlaSA) is defined as the set of ABIs and APIs exposed by the Fuchsia SDK. It is critical to maintain compatibility between subsequent releases of the Fuchsia Platform in order to ensure existing products and applications continue to operate correctly.
The Fuchsia Compatibility Test Suite (CTS) is one method of enforcing this compatibility. To learn more about how it works, please see the CTS RFC and the other pages on CTS in fuchsia.dev. This doc aims to help you understand what to do if a CTS test fails in Fuchsia CQ.
First, try to understand what part of the PlaSA has changed. If you already know, proceed to the next section.
CTS tests are designed to target a small part of the Fuchsia platform. The name and location of the failing test should be a good indication of what changed. In general (but not always), CTS tests will mirror their location in the SDK directory. Here are a few examples:
Tests located at ... | Are testing ... |
---|---|
//sdk/cts/fidl/fuchsia.diagnostics | //sdk/fidl/fuchsia.diagnostics |
//sdk/cts/pkg/memfs | //sdk/lib/memfs |
//sdk/cts/tools/package_manager | //src/sys/pkg/bin/pm |
You can find the name of the failing test by clicking on the core.x64-debug-cts
builder, and switching to the Test Results
tab.
If the name of the test gives no indication of what changed in the platform surface area, do a search of the CTS directory. The test code should live in a directory named after the PlaSA area it validates.
If all else fails, please feel free to reach out to the CTS team.
Note: TODO(http://fxbug.dev/85409): Test names may not clearly describe which PlaSA elements they exercise. Link here a map of CTS test -> PlaSA elements.
Once you've located the test code, you can use it to understand why your CL is causing it to fail. If it was an unintentional change, simply update your CL until the test no longer fails. If it was intentional, however, you have a couple of options on how to proceed:
The sections below will describe these options in more detail.
RFC-0002 describes our strategy for evolving the Fuchsia platform in a backwards-compatible way. Modifications to the public interface require a soft transition whenever possible: new elements are added, users of the old elements are migrated, then finally the old interface can be deprecated.
This is a multi-step process which may take weeks to complete. We are working to make this automatic and self-service, but please don't hesitate to reach out to the CTS team if you have any questions.
If a given CTS test is testing too much (e.g. some internal implementation details), or some other unique situation requires modifying the Fuchsia PlaSA more frequently than the CTS release window supports, then the CTS tests in the active CTS releases will need to be modified.
Note: TODO(fxbug.dev/86421): Needs design
Please reach out to the CTS team if this is required.