tree: 67f7a1b4e908be4fc7f1f64a749522c6ac6bd551 [path history] [tgz]
  1. contrib/
  2. lib/
  3. tests/
  4. add-update-source
  5. args
  6. build
  7. build-push
  8. BUILD.gn
  9. cipd.fx
  10. clean
  11. clean-build
  12. cp
  13. emu
  14. emu-remote
  15. exec
  16. fargo.fx
  17. fdt
  18. fetch-build-artifacts
  19. flash
  20. flash-remote
  21. gen
  22. get-build-artifacts
  23. get-build-dir
  24. get-device
  25. get-device-addr
  26. gn.fx
  27. goma_auth.fx
  28. goma_ctl.fx
  29. helpdoc
  30. is-package-server-running
  31. jiri.fx
  32. jq.fx
  33. klog
  34. list-boards
  35. list-build-artifacts
  36. list-devices
  37. list-packages
  38. list-products
  39. list-usb-disks
  40. log
  41. metrics
  42. metrics_testing.md
  43. mkzedboot
  44. multi
  45. netaddr
  46. netboot
  47. netboot-remote
  48. netls
  49. ninja.fx
  50. ninjatrace2json
  51. ota
  52. OWNERS
  53. pave
  54. pave-remote
  55. pave-zedboot
  56. pave-zedboot-remote
  57. push-package
  58. qemu
  59. README.md
  60. reboot
  61. run
  62. run-host-tests
  63. run-netboot
  64. run-test
  65. run-test-component
  66. scp
  67. self-test
  68. serial
  69. serve
  70. serve-remote
  71. serve-updates
  72. set
  73. set-build-dir
  74. set-device
  75. setup-macos
  76. setup-ufw
  77. sftp
  78. shell
  79. ssh
  80. status
  81. symbolize
  82. syslog
  83. test
  84. unset-device
  85. update
  86. update-if-in-base
  87. use
  88. vdl
  89. vendor
  90. wait
  91. zxdb.fx
tools/devshell/README.md

fx subcommands

Subcommands of fx are defined in several directories:

//tools/devshell contains core scripts that are part of fx workflow.

//tools/devshell/contrib contains scripts that have been contributed by project members that have other levels of support, ownership, or both. The OWNERS file in the contrib directory provides a pointer to the individuals supporting the scripts there.

//vendor/*/scripts/devshell contains scripts that are relevant only to the particular vendor and will have an ownership and support model documented there.

Subcommands can be implemented in a number of languages, but it is recommended to use bash at this time, so as to be able to consume some of the helpers provided by //tools/devshell/lib/vars.sh.

It is recommended that scripts be kept short and simple. Authoring large shell programs without a significant test plan can lead to hard to maintain tools. If there is a need to produce a more sophisticated program the recommended approach is to author a host tool program as part of the regular Fuchsia build, and only to wrap that program in a very slim way in a script. Examples of such cases can be found in fx pave and fx make-fuchsia-vol. A good rule of thumb here is that if a script only needs to launch and manage a one or a few sub-processes, then shell may be a fine language. If the program needs to perform any significant string manipulation or business logic, it is likely better authored in a language that provides more structural capabilities and standard library.

Consuming vars.sh and implementing subcommands

Most subcommands start with a pre-amble of this nature (paths vary slightly):

source "$(cd "$(dirname "${BASH_SOURCE[0]}")" >/dev/null 2>&1 && pwd)"/../lib/vars.sh || exit $?
fx-config-read

This pre-amble ensures that the devshell helpers are loaded, and then reads the active fx configuration from the user-selected Fuchsia build directory.

fx-config-read is required for most environment variables to be set, and is necessary for most scripts.

Helper functions

fx-config-read loads the current user configuration that is either defined by the fx configuration modulated by fx set and fx use, or by flags to fx such as --config and --dir. It is necessary as a pre-amble for any script that wants to consume build artifacts, as it defined most of the standard environment variables such as $FUCHSIA_BUILD_DIR.

fx-error and fx-warn simply print their arguments, prefixing them with ERROR: or WARNING: respectively. If the output stream supports color, then these outputs are colored.

fx-command-run and fx-command-exec execute another fx subcommand, for example, running fx-command-run shell will invoke the fx shell command. The run variant executes the subcommand in a subshell and the exec variant execs the subcommand, replacing the calling process. They both set a variable FX_CALLER to the caller script, which can be useful to switch behavior if the user executed the script directly or if it was executed by another fx script.

fx-command-help prints the command output for the currently running subcommand.

fx-export-device-address resolves the current device as currently configured by set-device, or detected via device-finder and exports:

  • FX_DEVICE_NAME the device name as either set using -d, fx set-device, or resolved by search.
  • FX_DEVICE_ADDR the device fuchsia address as resolved.
  • FX_SSH_ADDR the device address formatted as required for ssh(1). (IPv6 includes []).
  • FX_SSH_PORT the device SSH port if set via fx set-device.

get-device-name returns either the device name that the user has set with fx set-device or fx -d <device-name>. If the user has not set a default device, the command will run device discovery, and will return a discovered device name provided there is only one device discovered.

get-fuchsia-device-addr consumes get-device-name and returns the Fuchsia address of the device. The returned address is the “netstack address”, not the “netsvc address”.

get-device-addr-resource is the same as get-fuchsia-device-addr, except that it ensures the address is correctly formatted for use by tools such as ssh(1), i.e. for IPv6 addresses, the address is encased in [].

get-device-addr-url is the same as get-device-addr-resource except that it ensures that IPv6 scopes are appropriately percent-encoded.

fx-device-finder invokes the device-finder program with the given arguments. It reports an error to the user and exits if the program is not present in the build output. Most scripts should prefer one of the aforementioned get-* functions to perform related operations instead, as fx-device-finder usage will not be fx set-device aware.

The vars.sh script may define additional functions, however, they are considered internal and may change more often. Users can request additional helper functions by contacting the devshell owners, or by defining their own library scripts in contrib.

Environment variables

After a successful invocation of fx-config-read in a script, one would observe the following environment variables:

FUCHSIA_ARCH         - The current architecture selected (currently one of x64/arm64)
FUCHSIA_DIR          - The path to the root of the Fuchsia source tree
FUCHSIA_BUILD_DIR    - The path to the current Fuchsia build directory
HOST_OUT_DIR         - The path to the Fuchsia host-tools build directory
                         (usually $FUCHSIA_BUILD_DIR/host_$HOST_ARCH)
ZIRCON_BUILDROOT     - The path to the Zircon build directory
ZIRCON_TOOLS_DIR     - The path to the Zircon host-tools build directory.
FUCHSIA_OUT_DIR      - (deprecated) "$FUCHSIA_DIR/out"

fx-config-read and/or fx could set additional environment variables, but users should not rely on them - only the above list are to be preserved (unless marked deprecated).

Optional features

fx supports the definition of optional features that are enabled by default and can be disabled by the user for the duration of a single fx invocation.

These features can be used during the transition phase of Large Scale Changes that span across multiple commands. The potentially disruptive changes can be guarded behind an optional feature, so that users can be quickly unblocked by disabling the feature themselves.

Features have unique labels and shell commands can check if the given feature is enabled by using the is_feature_enabled method in //tools/devshell/lib/fx-optional-features.sh.

By default all optional features are enabled. If the user explicitly calls fx --disable=FEATURE ..., the feature is considered disabled for the duration of that call.

When the flag --disable=<FEATURE> is used in a fx call, fx exports an environmental variable named FUCHSIA_DISABLED_<FEATURE>, so all commands by default inherit it. Shell commands can verify if a feature is enabled by using the helper methods in fx-optional-features.sh, but non-shell commands, like Dart, can directly check for the value of the environmental variable. If FUCHSIA_DISABLED_<FEATURE> is set to “1”, “FEATURE” is disabled, otherwise it is enabled.

Documenting subcommands

As many fx subcommands delegate to sub-programs passing on flags directly to them, it is prohibitive to always be able to respond to the -h or --help flags. As such fx subcommands SHOULD implement -h and --help if possible, but this is not required.

It is required that all subcommands implement help documentation lines, which are defined as follows:

#### CATEGORY=Category name
### a short one-line (<70 character) description for the command lines
## usage: fx <subcommand> [-a|-b|-c] --foo ...
##
## Long descriptions, flags, and so on

The first line starting with ### is consumed by fx help to produce a list of subcommands with one-line descriptions of what the subcommand does. These lines should be kept short so as to render well under fx help.

Lines starting with ## are output when a user invokes fx help subcommand, and are used to provide full subcommand help output. The long form output should document all flags and provide fuller description of the subcommand behaviors as appropriate.

Lines starting with #### contain metadata. The following metadata fields are supported:

  • #### CATEGORY=Category: the subcommand is grouped under the specified category in the output of fx help. There's no enforcement on the name of the category, but whenever possible it should be one of the existing categories.

  • #### DEPRECATED: deprecated subcommands are not listed by default on fx help.

Where possible, a subcommand can use fx-command-help to print out the long-form help (defined by ## lines). Many subcommands implement -h and --help to invoke fx-command-help and this is recommended.

fx metadata files

When subcommands are scripts, documentation is embedded as comments in the scripts themselves. However, that's not always possible, for example for binaries produced by the build, such as fidldoc, prebuilt binaries like gn and symlinks like rustdoc and gen-cargo. In any case where metadata cannot be in the subcommand itself, fx looks for a metadata file with the .fx extension in the same directories where it looks for subcommands. If such a file exists, it represents a subcommand with the same name without the .fx extension.

<subcommand>.fx files follow the same format described in the previous section, with an optional metadata field:

  • #### EXECUTABLE=location_of_executable: points to the actual executable, which can be anywhere in the tree or in the build output. It can/must use the following variables to refer to known paths:

    • ${FUCHSIA_DIR}: root of the Fuchsia source tree
    • ${PREBUILT_3P_DIR}: location of the 3p prebuilts (usually ${FUCHSIA_DIR}/prebuilt/third_party)
    • ${HOST_PLATFORM}: platform of the host, used to compose prebuilt paths
    • ${HOST_TOOLS_DIR}: path of the host tools produced by the build

    Some examples of valid uses of EXECUTABLE in .fx files:

    • #### EXECUTABLE=${FUCHSIA_DIR}/.jiri_root/bin/cipd
    • #### EXECUTABLE=${PREBUILT_3P_DIR}/gn/${HOST_PLATFORM}/gn
    • #### EXECUTABLE=${FUCHSIA_DIR}/.jiri_root/bin/jiri
    • #### EXECUTABLE=${PREBUILT_3P_DIR}/ninja/${HOST_PLATFORM}/ninja

Testing

Testing shell subcommands

Subcommands that are shell scripts should be tested using the Bash test framework in //tools/devshell/tests/lib/bash_test_framework.sh, which provides facilities for mocking components and encapsulating the execution context in a temporary directory without any impact on the working tree.

Each test suite with one or more tests is a Bash script which name ends with _test in a subdirectory of //tools/devshell/tests.

To run shell tests, execute fx self-test <tests_script>. To find out what test scripts are available, run fx self-test without arguments and they will be listed at the bottom. To run all the tests from all the test scripts, use --all. Other sample invocations are described below:

fx self-test --all   # run all tests from all tests scripts
fx self-test subcommands    # run all tests scripts in //tools/devshell/tests/subcommands
fx self-test subcommands/fx_set_test   # run all tests in //tools/devshell/tests/subcommands/fx_set_test
fx self-test fx-internal/fx_test   # run all tests in //tools/devshell/tests/fx-internal/fx_test
fx self-test fx-internal/fx_test --test TEST_fx-subcommand-run   # run a single test from fx-internal/fx_test

To implement new shell test scripts, create a new file *_test in a subdirectory of //tools/devshell/tests using the Bash test framework documented in //tools/devshell/tests/lib/bash_test_framework.sh.

There are many examples in //tools/devshell/tests. The test framework is documented in the framework script.

Testing non-shell subcommands

Subcommands that are primarily non-shell, for example Rust or Dart, should have regular tests integrated with the Fuchsia build.

For example, the fx test subcommand is written in Dart and has tests defined in its BUILD.gn file.

Default device SSH keys

When you check out the Fuchsia tree, a jiri hook generates a default set of SSH credentials on your machine. These are used to pave Fuchsia devices and to connect to devices during development. You can find these keys stored in your home directory by default:

  • SSH private key (identity file): ~/.ssh/fuchsia_ed25519
  • SSH public key: ~/.ssh/fuchsia_ed25519.pub
  • List of public keys that allow connection to the Fuchsia device: ~/.ssh/fuchsia_authorized_keys

These paths are stored in a manifest in the root of your Fuchsia checkout. This file contains the absolute path to the Fuchsia private key and the absolute path to the Fuchsia-specific authorized_keys file, one on each line.

  • Manifest: //.fx-ssh-path

Previously, these credentials were stored inside the Fuchsia tree as //.ssh/pkey, //.ssh/pkey.pub, and //.ssh/authorized_keys. If SSH credentials are found in both locations and don't match, jiri update will fail. If you find yourself in this situation, remove one of the sets of keys and run jiri run-hooks.