These are tests to validate source-compatibility through various changes to FIDL libraries. They do not test binary-compatibility. Tests for compiled languages are only meant to be built, not executed, since source-compatibility issues show up at compile time.
This directory consists of:
[parent]-[target]-[change]format used in the FIDL ABI/API compatibility guide. These tests are generated using the tool above, and also each contain a generated README.md file describing the transition being tested.
Transitions that follow a specific pattern are sometimes given a specific name to make it easier to refer to these kinds of transitions when discussing them.
One set of terms we use is “FIDL assisted” and “source assisted”: when transitioning a FIDL library involves an initial state (before), an intermediate state (during), and a final state (after), depending on the bindings used and the kind of change made to the FIDL library, the transition is either FIDL-assisted or source-assisted.
In a FIDL-assisted transition, you change source code while the FIDL library is held in a transitional state (e.g., using the
Transitional attribute). For these transitions, we test four states:
In a source-assisted transition, you change the FIDL library while source code held is in a transitional state (e.g., using
default: in switch statements). This would lead to testing four states:
However, certain FIDL changes require a FIDL-assisted transition in some bindings and a source-assisted transition in others. Suppose there is a change in FIDL library L requiring a FIDL-assisted transition in bindings A and source-assisted in bindings B. The following steps would be taken:
All correct ways of interleaving the steps will have L and B both in the during state at some point. Therefore, although a FIDL during state is unnecessary for a pure source-assisted transition, we must include it in tests. Thus, we actually test 5 states for source-assisted transitions:
The transition example docs are each generated from individual source compatibility test cases. To update a file (e.g.
bits_member_add.md), you need to update the test that the file is generated from, and then regenerate the documentation.
The tests are located in
//src/tests/fidl/source_compatibility, and have a name matching the generated doc's filename. For example, the test corresponding to
test.json) using step number and binding and modify the value under the
"instructions"key. The value should be an array of strings, which will get rendered as a bulleted list in markdown.
src/tests/fidl/source_compatibility/gen/main.py regen bits-member-add. The argument of
regenshould be the name of the directory that the test is contained in.
fx format-codeto get rid of formatting differences, and verify the desired change
A test is declared by defining a test JSON configuration and adding a
source_compatibility_test target to the build (for examples, see the
test.json files within each test directory). This is done by running the source compatibility gen tool.
The rest of the commands in this README assume that this tool is aliased, e.g. by adding the following to your rc file:
A high level overview of the flow for creating a test using the tools is as follows:
scompat generate_test foo-rename-bar.
test.jsonfile containing the current state of the test, and a
BUILD.gnfile declaring a target for the test.
test.jsonfile open as you use the tool to verify the changes it makes. You can correct any typos or other mistakes that were provided to the tool by editing the JSON file, then running
test.json) is updated at each step, you can simply quit the tool when the transition is complete. It can also be helpful to build the test between adding steps to ensure that they are correct. Note however that the tool does not save the state until you complete the current step.
regencommand automatically regen the test-specific documentation).
//docs/development/languages/fidl/guides/compatibility/_toc.yamlto point to the newly generated documentation. You can do this by running
If you call the tool on an existing test (i.e. a directory containing a
test.json file), it will resume from where you last left off. In other words, the tool will only append new steps to the test, and cannot insert or otherwise edit existing steps. You must make edits manually by modifying the source files and
test.json file yourself, then run
scompat regen to regenerate the auxiliary files that are based on the
test.json, such as the README and GN sidecar file.
The test JSON structure is defined in
gen/types_.py, which contains a number of classes which correspond directly to the test JSON.
When a test fails to compile, the failure output will contain a path to a place somewhere in your out directory (what the path represents depends on the binding, e.g. for C++ the path is the path to the
.o file, whereas for Dart it's the path that the source file is copied to before building), that will tell you exactly which test/FIDL file/source file combination the error is coming from. For example, if you see the directory
somewhere in the path, you can deduce the following based on the ordering:
From this, you can deduce that the files in question are
protocol-event-remove/fidl/step_02_during.dart, and use that to debug the compile error.
Many tests are symmetrical: for example, removing a method consists of the same steps as adding a method, but backwards. Currently, the source compatibility tool provides a way to “bootstrap” a new test based on an existing one, by running it in reverse. For example, if you have a test
protocol-method-add and want to generate the symmetric
protocol-method-remove test based on it, you can run:
scompat generate_reverse protocol-method-add protocol-method-remove
This will create a copy of the method add test but in reverse. After generating the test, there are two additional manual steps:
protocolmethodremoveinside of the
"add method foo", would become
"remove method foo"in the reversed test.
scompat regen [test_name]) to regenerate the documentation based on any instruction changes.