This directory contains support files for using Bazel with the Fuchsia platform build.
The goal is to allow defining and building Bazel build targets in BUILD.bazel files in the platform source tree.
Bazel requires a workspace to run build commands. A Bazel workspace is identified by a MODULE.bazel file.
We create a synthesized Bazel workspace inside build output directory to execute Bazel build commands. The Fuchsia platform checkout itself is intentionally NOT a Bazel workspace yet.
The path to the synthesized Bazel workspace is determined by the content of //build/bazel/config/bazel_top_dir, referred to as ${BAZEL_TOPDIR} below. More specifically:
The ${BAZEL_TOPDIR} directory under the Ninja output directory is used to store all files related to the Bazel part of the build. By default this is out/default/gen/build/bazel from your Fuchsia source root.
The ${BAZEL_TOPDIR}/workspace directory is used as the Bazel workspace to invoke all bazel commands.
The ${BAZEL_TOPDIR}/output_base directory is used to place all Bazel outputs (including external repositories).
The ${BAZEL_TOPDIR}/bazel wrapper script is used to launch a known prebuilt Bazel binary with the right options to support this setup, and other requirements of the platform build. It's a symlink to //build/bazel/wrapper.bazel.sh
The above mentioned contents in ${BAZEL_TOPDIR} are generated by running fx set or fx gen. The workspace subdirectory has a specific layout that mirrors the Fuchsia source tree with a few exceptions:
A top-level BUILD.bazel file, which is by default a symlink to //build/bazel/toplevel.BUILD.bazel in the source tree. This is the file you should modify to add new top-level targets in the Bazel build graph.
A top-level MODULE.bazel file, which are by default a symlink to //build/bazel/toplevel.MODULE.bazel in the source tree. This is the file you should modify to add new external repositories to the Bazel project.
A top-level auto-generated .bazelrc file to configure Bazel. Note that this does not support --config=fuchsia_x64 and --config=fuchsia_arm64 as per the Fuchsia Bazel SDK. Also note that the user's own bazel configuration file (e.g. $HOME/.bazelrc) will always be ignored. See Configurations for more details.
Symlinks to all top-level entries in the Fuchsia source tree, except for out. This way, any source or configuration file will appear at the same exact location during the Bazel and Ninja builds.
In other words, if you add a file at ${FUCHSIA_DIR}/src/foo/BUILD.bazel, it will be visible and will define a Bazel package at //src/foo:.
The out directory is NOT symlinked intentionally to ensure that Ninja outputs are not visible by default. Instead, these are exposed to Bazel on a case-by-case basis using special bazel_input_xxx() template calls in BUILD.gn files. For more details, see the corresponding section below.
Other Bazel external repositories are generated through repository rules in MODULE.bazel, for example the @prebuilt_clang repository will provide C++ toolchain definitions that use the prebuilt Clang toolchain available from the Fuchsia checkout.
This is intentionally different from the @fuchsia_clang repository generated by //build/bazel_sdk/bazel_rules_fuchsia, since this allows experimenting with C++ toolchain definitions, build variants and PIE-optimized machine code generation without conflicts with sdk-integration development.
The Bazel build generates an external repository named @prebuilt_clang that mirrors the content of the host Clang toolchain, and augments it with Bazel-specific files. These are needed to define C++ toolchain instances and configurations.
C++ Toolchain selection is performed using the new Bazel Platforms toolchain, which is very different to the traditional use of --crosstool_top, which is why using this option will not work.
Note that this is distinct from sdk-integration's own @fuchsia_clang external repository, which defines a C++ toolchain that generates Fuchsia binaries (while @prebuilt_clang is used to generate host binaries and experiment with build variants and PIE-optimized executables, without conflicts).
IMPORTANT: This feature is purely for developer convenience during local development and nothing should depend on these links in our build rules or scripts. In particular, these directories never exist on infra builders.
Several convenience symlinks in your Fuchsia source tree are created by fx set to access the Bazel workspace, external repositories and artifacts more easily. Namely:
bazel-workspace points to the Bazel workspace directory.
bazel-bin and bazel-out have the same role as with regular Bazel, but will only point to directories corresponding to the latest Bazel build invocation. Keep in mind that a single fx build <target> command might end up invoking several different Bazel build commands.
bazel-repos points to the directory where all external repositories are stored. This is mostly useful to the build team to inspect that repository rules generated the correct output.
These symlinks targets will be updated on each fx set, fx gen, fx use or fx change-build-dir command.
The Bazel workspace provides by default a @fuchsia_sdk repository that exposes Fuchsia SDK atoms from the source tree to the Bazel graph. However, this workspace cannot be used, except for queries, unless the in-tree IDK has been built with Ninja. Building the in-tree IDK is a lengthy operation that can easily take several minutes on powerful workstations, and even longer on less capable machines.
It is possible to avoid this dependency, by not populating @fuchsia_sdk with Ninja artifacts. This allows building host Bazel targets (e.g. host tools) directly, just after fx set or fx gen is called.
There are three ways to use this mode:
In GN bazel_action() target definitions by setting no_sdk = true to indicate that the bazel targets do not require Fuchsia SDK dependencies at all.
When invoker fx build to build Bazel targets, specify --host, for example fx build --host @//build/tools/formatjson5.
When invoking fx bazel directly, specify the host platform using --config=host, for example fx bazel build --config=host //build/tools/formatjson5.
It is possible to safely switch between normal and no-sdk variants operations. For example:
# Clean build directory, and set a new build configuration. fx set minimal.x64 --with //build/bazel:tests fx clean # Build and run a series of host tests directly from Bazel # This does not invoke Ninja at all. fx bazel test --config=host //build/bazel/host_tests/... # Build a host hello_world program with Bazel, invoked from # a Ninja action. Because this target definition uses `no_sdk=true` # this is fast, as it doesn't require building the IDK. fx build //build/bazel/examples/hello_no_sdk # Build a similar program for Fuchsia, this will take several # minutes because it requires building the IDK first, even though # the program does not depend on any SDK atom. :-/ fx build //build/bazel/examples/hello_world
The reasons why there are not top-level MODULE.bazel or BUILD.bazel files provided in ${FUCHSIA_DIR} are that:
Some top-level Bazel files (e.g. .bazelrc) contain values that always must be auto-generated or adjusted on each Bazel invocation. This can only be performed through wrapper scripts, not direct invocation of the tool.
Developers cannot call Bazel directly from the Fuchsia source tree (which otherwise would likely fail with very confusing error messages).
Bazel's traditional command line interface to configure Bazel is not compatible with the requirements of the Fuchsia platform build. In particular, the Fuchsia Bazel SDK samples use --config=fuchsia_arm64 to specify the target device architecture, but this will not work here.
Finally, Bazel support in the Fuchsia platform build is still very experimental, and is best considered an implementation detail, hidden from developer workflows.
Similarly, the reason why Bazel build artifacts are placed under ${BAZEL_TOPDIR}/output_base, instead of using the standard location under the user's home directory (e.g. $HOME/.cache/bazel/) are:
This allows fx clean to properly remove previous build artifacts.
This increases the chance of using hard-links when copying Bazel build outputs to the Ninja output directory, since in many setups, $HOME lives in a different partition / mount point than the Fuchsia checkout.
This prevents filling up the user-specific directory with hundreds of GiBs of build artifacts that can be hard to clean up properly (e.g. when deleting a Fuchsia checkout directory manually with rm -rf).
In particular, a Bazel module extension is used to generate a repository named @prebuilt_clang that provides C++ toolchain instances and configurations for the build.
The Fuchsia platform build also relies on the new Platforms feature, which impacts how toolchain selection happens inside a given build configuration. See //build/bazel/platforms/BUILD.bazel for more details.
The bazel_build_action template, defined in //build/bazel/bazel_build_action.gni, is used to create a GN action that will run a bazel build <targets> command in the Bazel workspace.
If such a Bazel target depends on a Ninja-generated output file, this must be expressed by a dependency on a bazel_input_xxx() target, as explained in the later section,
After the Bazel build command completes, its outputs are copied from the non-deterministic location in the Bazel output base into a stable location under target_out_dir or target_gen_dir (as required by GN for all actions).
For a concrete example, see //build/bazel/examples/hello_test/BUILD.gn that contains a working target that invokes a Bazel build command (that simply copies an input file into a Bazel output), then verify that it worked properly.
For host targets defined in Bazel, you can build them directly with fx build, for example fx build --host @//build/tools/formatjson5. Build outputs will be available in bazel-bin and bazel-out
IMPORTANT:
fx bazelis for debugging purposes only, there is no guarantee that your Bazel workspace is setup properly to run the Bazel command you specified, for example it can be missing contents of the SDK, or outputs from Ninja that your Bazel command need to consume as inputs.
For debugging only, the fx bazel wrapper tool is provided to launch a Bazel command inside the Fuchsia workspace file. This will always update the Bazel workspace if needed.
For example, use fx bazel version to print information about the version number, of fx bazel info workspace to print the absolute path to the workspace.
And fx bazel build ... or fx bazel query ... commands will work as expected.
GN target outputs (i.e. Ninja build artifacts) can be exposed to the Bazel graph as inputs through bazel_input_file() and bazel_input_directory(), which expose the outputs of other GN targets as filegroups in the special @gn_targets external repository.
A GN bazel_action() target must be defined to invoke Bazel, through a script, and must list the bazel_input_xxx() in its dependencies to ensure that the right Ninja outputs are exposed to the Bazel graph before the Bazel command runs.
@gn_targets repositoryHere's a simple example that uses the first scheme. It allows a Bazel target to process the output of a GN action, and copy the result back to the Ninja build directory through the //src/lib:process_foo GN target.
On the GN side:
# From //src/lib/BUILD.gn, evaluated in the default toolchain context: # An action that generates one or more output files. action("foo") { outputs = [ ... ] } # A target that exposes the outputs of :foo as # as the @gn_targets//src/lib:foo filegroup() bazel_input_file("foo.bazel_input") { generator = ":foo" } # A target that invokes a Bazel target to process the # foo outputs. The result is copied to $BUILD_DIR/obj/src/lib/foo.final bazel_action("process_foo") { command = "build" deps = [ ":foo.bazel_input" ] bazel_targets = [ ":foo_processor" ] copy_outputs = { bazel = "{{BAZEL_TARGET_OUT_DIR}}/foo.processed_by_bazel" ninja = "foo.final" } }
And on the Bazel side.
# From //src/lib/BUILD.bazel genrule( name = "foo_processor", srcs = [ "@gn_targets//src/lib:foo" ], outs = [ "foo.processed_by_bazel" ], command = "process.sh $< $@", )
Notice the //src/lib:foo.bazel_input GN target definition. This does not build anything, but records information about the outputs of the generator target //src/lib:foo.
The //src/lib:process_foo GN target depends on it, which will force the content of the special @gn_targets repository to be automatically updated before invoking Bazel to reflect the target's dependencies.
In this case, because //src/lib:process_foo depends on //src/lib:foo.bazel_input in the GN graph, the Bazel @gn_targets//src/lib:foo filegroup will be defined, grouping the outputs of the GN //src/lib:foo target, built or updated by Ninja before Bazel is invoked.
Note that all Ninja outputs are accessed in Bazel through filegroups named from the GN target label that exposes it. I.e. one cannot access files using a Ninja artifact path such as @gn_targets//obj/src/lib/foo.out.
The bazel_input_file() template requires a generator argument that must point to a GN target that generates Ninja output files.
The corresponding Bazel filegroup will be defined as @gn_targets//{package_name}:{bazel_name} where:
{bazel_name} matches the name of the generator target itself, but this value can be overridden using the optional gn_targets_name argument.
{bazel_package} matches the directory of the generator target itself, if it is defined in the default GN toolchain context. Otherwise, it will include a toolchain_{toolchain_name}/ prefix as well.
Note that the directory and names of the bazel_input_file() target itself does not impact the content of @gn_targets at all.
Hence the following examples:
# From //src/lib/BUILD.gn action("foo") { ... } # This creates `@gn_targets//src/lib:foo` bazel_input_file("foo_outputs") { generator = ":foo" } # This creates `@gn_targets//src/lib:foo_alt`, by overriding the # name explicitly. bazel_input_file("foo_outputs_alt") { generator = ":foo" gn_targets_name = "foo_alt" } if (current_toolchain == default_toolchain) { # This creates @gn_targets//toolchain_host_x64/src/lib:foo bazel_input_file("foo_host_outputs") { generator = ":foo($host_toolchain)" # Because the generator is in a different toolchain context, # its `outputs` argument must be provided (see next section). outputs = [ get_label_info(generator, "target_out_dir") + "/foo.out" ] } }
If one defines several bazel_input_file() with the same filegroup package and name, Bazel will complain about multiply defined targets in the corresponding BUILD.bazel file (which can be inspected, as comments indicate which exact GN target defined them).
By default, bazel_input_file() exposes all outputs of the generator target, but this can only work if the following conditions apply:
action() (i.e. not a group() or an executable()).BUILD.gn file.Otherwise, specifying the list of outputs using the outputs argument is required. GN will try to print a user-friendly error message explaining the situation. An example of explicit outputs:
# From //src/lib/BUILD.gn action("foo") { ... } # From //src/lib/bar/BUILD.gn # Only expose the (first) output of //src/lib:foo as @gn_targets//src/lib:foo # The fact that this target is defined under //src/lib/bar/ does not matter. bazel_input_file("foo.bazel_input") { generator = "//src/lib:foo" _foo_output_dir = get_label_info(generator, "target_out_dir") outputs = [ "${_foo_output_dir}/output", ] }
Note that this generates the filegroup as @gn_targets//src/lib:foo, and not as @gn_targets//src/lib/bar:foo.
This is why defining bazel_input_file() targets in the same BUILD.gn file as the generator target is recommended, though not required.
It is an error to list a directory as an output in a bazel_input_file(), as this will may result in incremental build errors. There is no easy way to detect this from GN / Ninja, but one can use the bazel_input_directory() GN template to expose directory Ninja outputs to Bazel.
This creates a Bazel filegroup that uses a glob() statement, under the hood, to ensure that all files from the directory are visible from the Bazel sandbox / command execution environment, and carry dependency information properly across the GN / Bazel graph boundaries.
Bazel command-line configurations are defined in //build/bazel/templates/template.bazelrc, which produces ${BAZEL_TOPDIR}/.bazelrc. Configurations are invoked as --config=NAME.
Sharing build results can be helpful for triaging and reproducing issues. Use one of:
--config=sponge streams build event data to the Sponge service.--config=resultstore streams build event data to the ResultStore service.Remote building can speed up builds by 1) offloading work remotely and 2) caching.
--config=remote enables remote build for many actions using RBE.You can invoke build/bazel/scripts/test-all.py to verify that everything works properly. This is used to verify regressions during development of Bazel support in the platform build, what is does is:
Invoke build/bazel/scripts/prepare-fuchsia-checkout.py, unless the --skip-prepare option is used.
Invoke fx clean unless --skip-clean is used.
Setup the Bazel workspace, then run a series of tests to verify that things work properly.
Always try to add new tests when introducing new features under //build/bazel/, and run this script when changing its implementation (for now this is all manual, but will likely be automated in CQ in the future).
Most non-remote bazel build actions are launched in a sandbox, whose content disappears after the build, even in case of failure. It is possible to set FUCHSIA_DEBUG_BAZEL_SANDBOX=1 in the environment before invoking fx build to tell Bazel to preserve the sandboxes.
Note that setting this variable makes Bazel much more chatty, and too many stale sandboxes can become a problem, so this is best used when encountering a sandboxing-related issue, which most of the time come from inputs missing from the sandbox due to missing dependency.
Also note that this does not affect remote actions (i.e. if you have RBE enabled). It is however possible to force a specific Bazel target to run in a local sandbox run by using adding the no-remote string to its tags attribute, as in:
cc_library( name = "bin", srcs = [ ... ], deps = [ ... ], tags = [ "no-remote" ], )
To help debug hermeticity issues that happen within repository rules, the Bazel workspace is configured to save a log of repository-related events to a file located under ${BAZEL_TOPDIR}/logs/workspace-events.log, this is a binary proto file (see https://bazel.build/remote/workspace) that can be converted to text automatically with //build/bazel/scripts/parse-workspace-event-log.py which can be invoked directly from your Fuchsia directory.
The script will automatically find the log for the latest command and dump its content to text on stdout, or to a file if the --output=FILE option is used.
The log files are rotated, up to 3 older revisions are stored in the ${BAZEL_TOPDIR}/logs directory so you can compare them if possible.