This is the integration tests for a number of system tests.
In order to build the system tests, add this to your fx set
:
% fx set ... \ --with //src/sys/pkg:e2e_tests \ --with //src/testing/sl4f \ --with //src/sys/bin/start_sl4f \ --args='core_realm_shards = [ "//src/testing/sl4f:sl4f_core_shard" ]' % fx build
Next, you need to authenticate against luci to be able to download build artifacts. First, authenticate against cipd with:
% cipd auth-login
Next, install chromium's depot_tools
by following these instructions. Then, login to luci by running:
% cd depot_tools % ./luci-auth login -scopes "https://www.googleapis.com/auth/userinfo.email https://www.googleapis.com/auth/devstorage.read_write" ...
Now you should be able to run the tests against any device that is discoverable by ffx target list
.
This tests Over the Air (OTA) updates. At a high level, it:
You should be able to run the tests with:
% $(fx get-build-dir)/host_x64/system_tests_upgrade \ --ssh-private-key $(ffx config get ssh.priv | tr -d '"') \ --builder-name fuchsia/global.ci/core.x64-release-nuc_in_basic_envs \ --fuchsia-build-dir $(fx get-build-dir)
This will run through the whole test paving the build to the latest version available from the specified builder, then OTA-ing the device to the local build directory.
The Upgrade Tests also support reproducing a specific build. To do this, determine the build ids from the downgrade and upgrade builds, then run:
% $(fx get-build-dir)/host_x64/system_tests_upgrade \ --ssh-private-key $(ffx config get ssh.priv | tr -d '"') \ --build-id 123456789... \ --build-id 987654321...
Or you can combine these options:
% $(fx get-build-dir)/host_x64/system_tests_upgrade \ --ssh-private-key $(ffx config get ssh.priv | tr -d '"') \ --build-id 123456789... \ --fuchsia-build-dir $(fx get-build-dir)
There are more options to the test, to see them all run $(fx get-build-dir)/host_x64/system_tests_upgrade -- -h
.
The system tests support running reboot tests, where a device is rebooted a configurable number of times, or errs out if a problem occurs. This can be done by running:
% $(fx get-build-dir)/host_x64/system_tests_reboot \ --ssh-private-key $(ffx config get ssh.priv | tr -d '"') \ --fuchsia-build-dir $(fx get-build-dir)
Or if you want to test a build, you can use:
--builder-name fuchsia/global.ci/core.x64-release-nuc_in_basic_envs
, to test the latest build published by that builder.--build-id 1234...
to test the specific build.When running the system tests, it's helpful to capture the serial logs, and system logs, and the test output to a file in order to triage any failures. This is especially handy when cycle testing. To simplify the setup, the system-tests come with a helper script run-test
that can setup a tmux
session for you. You can run it like this:
% ${FUCHSIA_DIR}/src/sys/pkg/tests/system-tests/bin/run-test \ -o ~/logs \ --tty /dev/ttyUSB0 \ $(fx get-build-dir)/host_x64/system_tests_upgrade \ --builder-name fuchsia/global.ci/core.x64-release-nuc_in_basic_envs \ --fuchsia-build-dir $(fx get-build-dir)
This will setup a tmux
with 3 windows, one for the serial session on /dev/ttyUSB0
, one for the system logs, and one for the test. All output from the tmux
windows will be saved into ~/logs
.
See the run-test --help
for more options.
At the moment, the build script fx qemu
does not bring up a configuration that can be OTA-ed. Until this is implemented, the bin/
directory contains some helper scripts that bring up an OTA-able Fuchsia emulator. Follow these instructions to it.
The create-emu
script will create a Fuchsia EFI image:
% ./bin/create-emu some/image
This image can then be used by running:
% fx qemu --uefi -D some/path/to/image
Each script supports --help
to see other supported flags.
These scripts wrap fx make-fuchsia-vol
to create the Fuchsia image, and //zircon/scripts/run-zircon
to run QEMU.
Once the VM has finished paving, you can then use it with the upgrade tests:
% $(fx get-build-dir)/host_x64/system_tests_upgrade \ --ssh-private-key $(ffx config get ssh.priv | tr -d '"') \ --builder-name fuchsia/global.ci/workstation_eng.x64-release-e2e-isolated \ --fuchsia-build-dir $(fx get-build-dir) \ --device fuchsia-5254-0063-5e7a
Note that your QEMU device will always be named fuchsia-5254-0063-5e7a
. Explicitly naming the device for the test will help prevent accidental upgrades to your other devices.