End to end product tests for fuchsia products are located here.
Unlike other kinds of tests, end to end product tests are specific to a product, and therefore must not be included in the global tests
build target mandated by the source tree layout. Instead, end to end product tests are included directly in the build configuration of their products in //products.
Please also add an OWNERS
file so that you can edit your test at will. If there are run-time dependencies needed for the test you can add them to the /bundles:end_to_end_deps
group.
Because end to end tests are tied to a specific product or even architecture, you should review the test's README.md to see which product or special instructions are needed to run the test.
To run an end to end test use the run-e2e-tests
command:
$ fx set <product>.<arch> && fx build $ fx run-e2e-tests name_of_the_test
Where name_of_the_test
is the name of the dart_test rule (like sl4f_test
).
*_packages
grouplacewing-admin@google.com
to create the builderpython_mobly_test
rule in BUILD.gn to include appropriate BOARDS (based on what all the boards this test need to be run in infra) in environments
field (example)lacewing-builder-users@google.com
group so that you will be notified whenever any of the tests in the builder is failed. You can then check for your test result in that run and triage if it has failed.Please reach out to Lacewing team if you need help with this one.
Lacewing test case can be run in Infra in of the following builder types:
Use the following approach in deciding whether to run the test case in CQ/CI/FYI:
Once your test case is ready to be promoted to CI/CQ:
stable
Lacewing User Tests Infra Groups exists, and that test group is already enabled in a CQ builder, simply move your test target from unstable
to stable
.Based on this we have created the following:
<PRODUCT>_<BOARD>_<STABILITY>[ |_sl4f]_tests
<PRODUCT>
- The product that the tests require to run on - e.g. “core”, “workbench”.<BOARD>
- The board that the tests require to run on - e.g. “emulator”, “nuc”, “vim3”.<STABILITY>
- Whether tests are stable or flaky - “stable” or “unstable”. All newly added tests must be added to the “unstable” groups until 200 passing runs in infra FYI builder have been observed, after which they may be promoted to “stable” groups.[ |_sl4f]
- If tests require SL4F server then include “_sl4f”. Otherwise, leave it empty.core_emulator_stable_tests
workbench_vim3_unstable_tests
<STABILITY>
informs whether a group is potential to run in CI/CQ (as it depends on <BOARD>
also).[ |_sl4f]
informs whether a test group can be run on certain products.<BOARD>
informs whether a test group can be run on certain boards.<PRODUCT>_<BOARD>_<STABILITY>[ |_sl4f]_packages
<PRODUCT>.<BOARD>-[debug|release]-lacewing[-users| ]-[ |staging|ci]
-users
will be used in the builder name-staging
or -ci
)core.x64-debug-lacewing
- CQ builder to run stable emulator and NUC testscore.x64-debug-lacewing-users-staging
- FYI builder to run unstable emulator and NUC testscore.vim3-debug-lacewing-users-staging
- FYI builder to run unstable VIM3 tests