This directory contains experimental support files for using Bazel with the Fuchsia platform build. Nothing here is final and may break at any time for now.
Have a full Fuchsia platform checkout from Jiri, and run the following on a Linux build machine (MacOS is not supported at the moment!)
Now that you‘ve been warned, here’s how this is supposed to work:
The BAZEL_TOPDIR directory under the Ninja output directory is used to store all files related to the Bazel part of the build. Its default value is out/default/gen/build/bazel.
The ${BAZEL_TOPDIR}/workspace directory is used as the Bazel workspace for all bazel commands invoked from Ninja.
The ${BAZEL_TOPDIR}/output_base directory is used to place all Bazel outputs (including external repositories).
The ${BAZEL_TOPDIR}/bazel wrapper script is used to launch a known prebuilt Bazel binary with the right options to support this setup, and other requirements of the platform build.
All four are generated by running fx build bazel_workspace (which ends up calling targets in //build/bazel/BUILD.gn). The workspace content has a specific layout that mirrors the Fuchsia source tree with a few exceptions:
A top-level BUILD.bazel symlink to //build/bazel/toplevel.BUILD.bazel in the source tree. This is the file you should modify to add new top-level targets in the Bazel build graph.
A top-level WORKSPACE.bazel generated from //build/bazel/templates/template.WORKSPACE.bazel in the source tree. This is the file you should modify to add new external repositories to the Bazel project.
A top-level auto-generated .bazelrc file to configure Bazel. Note that this does not support --config=fuchsia_x64 and --config=fuchsia_arm64 are per the Fuchsia Bazel SDK. Also note that the user's own bazel configuration file (e.g. $HOME/.bazelrc) will always be ignored. See Configurations for more details.
Symlinks to all top-level entries in the Fuchsia source tree, except for out. This way, any source or configuration file will appear at the same exact location during the Bazel and Ninja builds.
In other words, if you add a file at $FUCHSIA_DIR/src/foo/BUILD.bazel, it will be visible and will define a Bazel package at //src/foo:.
The out directory is not symlinked intentionally to ensure that Ninja outputs are not visible by default. Instead, these are exposed to Bazel on a case-by-case basis using special bazel_input_xxx() template calls in BUILD.gn files. For more details, see the corresponding section below.
Other Bazel external repositories are generated through repository rules, for example the @prebuilt_clang repository will provide C++ toolchain definitions that use the prebuilt Clang toolchain available from the Fuchsia checkout.
This is intentionally different from the @fuchsia_clang repository generated by third_party/sdk-integration, since this allows experimenting with C++ toolchain definitions, build variants and PIE-optimized machine code generation without conflicts with sdk-integration development.
The GN bazel_build_action() template can be used to invoke a bazel build ... command from GN, and copy its outputs to a GN-compatible location. See //build/bazel/bazel_build_action.gni for details.
The //build/bazel/tests/build_action/ directory contains a working example, which invokes a Bazel build action from GN, then verifies the output.
For debugging only, the fx bazel wrapper tool is provided to launch a Bazel command inside the Fuchsia workspace file. This will always update the Bazel workspace if needed.
For example, use fx bazel version to print information about the version number, of fx bazel info workspace to print the absolute path to the workspace.
And fx bazel build ... or fx bazel query ... commands will work as expected.
The reasons why there are not top-level WORKSPACE.bazel or BUILD.bazel files provided in $FUCHSIA_DIR are that:
The content of WORKSPACE.bazel contain multiple values that depend on the host machine type, and soon on the actual Jiri manifest or git supermodule hierarchy, and thus must always be auto-generated.
Developers cannot call Bazel directly from the Fuchsia source tree (which otherwise would likely fail with very confusing error messages).
Bazel's traditional command line interface to configure Bazel is not compatible with the requirements of the Fuchsia platform build. In particular, the Fuchsia Bazel SDK samples use --config=fuchsia_arm64 to specify the target device architecture, but this will not work here.
Finally, Bazel support in the Fuchsia platform build is still very experimental, and is best considered an implementation detail, hidden from developer workflows.
Similarly, the reason why Bazel build artifacts are placed under ${BAZEL_TOPDIR}/output_base, instead of using the standard location under the user's home directory (e.g. $HOME/.cache/bazel/) are:
This allows fx clean to properly remove previous build artifacts.
This increases the chance of using hard-links when copying Bazel build outputs to the Ninja output directory, since in many setups, $HOME lives in a different partition / mount point than the Fuchsia checkout.
This prevents filling up the user-specific directory with hundreds of GiBs of build artifacts that can be hard to clean up properly (e.g. when deleting a Fuchsia checkout directory manually with rm -rf).
In particular, a Bazel module extension is used to generate a repository named @prebuilt_clang that provides C++ toolchain instances and configurations for the build.
The Fuchsia platform build also relies on the new Platforms feature, which impacts how toolchain selection happens inside a given build configuration. See //build/bazel/platforms/BUILD.bazel for more details.
GN target outputs (i.e. Ninja build artifacts) can be exposed to the Bazel graph as inputs through bazel_input_file() and bazel_input_directory(), which expose the outputs of other GN targets as filegroups in the special @gn_targets external repository.
A GN bazel_action() target must be defined to invoke Bazel, through a script, and must list the bazel_input_xxx() in its dependencies to ensure that the right Ninja outputs are exposed to the Bazel graph before the Bazel command runs.
@gn_targets repositoryHere's a simple example that uses the first scheme. It allows a Bazel target to process the output of a GN action, and copy the result back to the Ninja build directory through the //src/lib:process_foo GN target.
On the GN side:
# From //src/lib/BUILD.gn, evaluated in the default toolchain context: # An action that generates one or more output files. action("foo") { outputs = [ ... ] } # A target that exposes the outputs of :foo as # as the @gn_targets//src/lib:foo filegroup() bazel_input_file("foo.bazel_input") { generator = ":foo" } # A target that invokes a Bazel target to process the # foo outputs. The result is copied to $BUILD_DIR/obj/src/lib/foo.final bazel_action("process_foo") { command = "build" deps = [ ":foo.bazel_input" ] bazel_targets = [ ":foo_processor" ] copy_outputs = { bazel = "{{BAZEL_TARGET_OUT_DIR}}/foo.processed_by_bazel" ninja = "foo.final" } }
And on the Bazel side.
# From //src/lib/BUILD.bazel genrule( name = "foo_processor", srcs = [ "@gn_targets//src/lib:foo" ], outs = [ "foo.processed_by_bazel" ], command = "process.sh $< $@", )
Notice the //src/lib:foo.bazel_input GN target definition. This does not build anything, but records information about the outputs of the generator target //src/lib:foo.
The //src/lib:process_foo GN target depends on it, which will force the content of the special @gn_targets repository to be automatically updated before invoking Bazel to reflect the target's dependencies.
In this case, because //src/lib:process_foo depends on //src/lib:foo.bazel_input in the GN graph, the Bazel @gn_targets//src/lib:foo filegroup will be defined, grouping the outputs of the GN //src/lib:foo target, built or updated by Ninja before Bazel is invoked.
Note that all Ninja outputs are accessed in Bazel through filegroups named from the GN target label that exposes it. I.e. one cannot access files using a Ninja artifact path such as @gn_targets//obj/src/lib/foo.out.
The bazel_input_file() template requires a generator argument that must point to a GN target that generates Ninja output files.
The corresponding Bazel filegroup will be defined as @gn_targets//{package_name}:{bazel_name} where:
{bazel_name} matches the name of the generator target itself, but this value can be overridden using the optional gn_targets_name argument.
{bazel_package} matches the directory of the generator target itself, if it is defined in the default GN toolchain context. Otherwise, it will include a toolchain_{toolchain_name}/ prefix as well.
Note that the directory and names of the bazel_input_file() target itself does not impact the content of @gn_targets at all.
Hence the following examples:
# From //src/lib/BUILD.gn action("foo") { ... } # This creates `@gn_targets//src/lib:foo` bazel_input_file("foo_outputs") { generator = ":foo" } # This creates `@gn_targets//src/lib:foo_alt`, by overriding the # name explicitly. bazel_input_file("foo_outputs_alt") { generator = ":foo" gn_targets_name = "foo_alt" } if (current_toolchain == default_toolchain) { # This creates @gn_targets//toolchain_host_x64/src/lib:foo bazel_input_file("foo_host_outputs") { generator = ":foo($host_toolchain)" # Because the generator is in a different toolchain context, # its `outputs` argument must be provided (see next section). outputs = [ get_label_info(generator, "target_out_dir") + "/foo.out" ] } }
If one defines several bazel_input_file() with the same filegroup package and name, Bazel will complain about multiply defined targets in the corresponding BUILD.bazel file (which can be inspected, as comments indicate which exact GN target defined them).
By default, bazel_input_file() exposes all outputs of the generator target, but this can only work if the following conditions apply:
action() (i.e. not a group() or an executable()).BUILD.gn file.Otherwise, specifying the list of outputs using the outputs argument is required. GN will try to print a user-friendly error message explaining the situation. An example of explicit outputs:
# From //src/lib/BUILD.gn action("foo") { ... } # From //src/lib/bar/BUILD.gn # Only expose the (first) output of //src/lib:foo as @gn_targets//src/lib:foo # The fact that this target is defined under //src/lib/bar/ does not matter. bazel_input_file("foo.bazel_input") { generator = "//src/lib:foo" _foo_output_dir = get_label_info(generator, "target_out_dir") outputs = [ "${_foo_output_dir}/output", ] }
Note that this generates the filegroup as @gn_targets//src/lib:foo, and not as @gn_targets//src/lib/bar:foo.
This is why defining bazel_input_file() targets in the same BUILD.gn file as the generator target is recommended, though not required.
It is an error to list a directory as an output in a bazel_input_file(), as this will may result in incremental build errors. There is no easy way to detect this from GN / Ninja, but one can use the bazel_input_directory() GN template to expose directory Ninja outputs to Bazel.
This creates a Bazel filegroup that uses a glob() statement, under the hood, to ensure that all files from the directory are visible from the Bazel sandbox / command execution environment, and carry dependency information properly across the GN / Bazel graph boundaries.
The bazel_build_action() template, defined in //build/bazel/bazel_build_action.gni is used to create a GN action that will run a bazel build <targets> command in the Bazel workspace.
If such a Bazel target depends on a Ninja-generated output file, this must be expressed by a dependency on a bazel_input_xxx() target, as explained in the previous section,
After the Bazel build command completes, its outputs are copied from the non-deterministic location in the Bazel output base into a stable location under target_out_dir or target_gen_dir (as required by GN for all actions).
For a concrete example, see //build/bazel/tests/build_action/ that contains a working target that invokes a Bazel build command (that simply copies an input file into a Bazel output), then verify that it worked properly.
The Bazel build generates an external repository named @prebuilt_clang that mirrors the content of the host Clang toolchain, and augments it with Bazel-specific files. These are needed to define C++ toolchain instances and configurations.
C++ Toolchain selection is performed using the new Bazel Platforms toolchain, which is very different to the traditional use of --crosstool_top, which is why using this option will not work.
Note that this is distinct from sdk-integration's own @fuchsia_clang external repository, which defines a C++ toolchain that generates Fuchsia binaries (while @prebuilt_clang is used to generate host binaries and experiment with build variants and PIE-optimized executables, without conflicts).
Bazel command-line configurations are defined in //build/bazel/templates/template.bazelrc, which produces ${BAZEL_TOPDIR}/.bazelrc. Configurations are invoked as --config=NAME.
Sharing build results can be helpful for triaging and reproducing issues. Use one of:
--config=sponge streams build event data to the Sponge service.--config=resultstore streams build event data to the ResultStore service.Remote building can speed up builds by 1) offloading work remotely and 2) caching.
--config=remote enables remote build for many actions using RBE.--config=gcertauth uses your LOAS credentials to authenticate for using the aforementioned build services.You can invoke build/bazel/scripts/test-all.py to verify that everything works properly. This is used to verify regressions during development of Bazel support in the platform build, what is does is:
Invoke build/bazel/scripts/prepare-fuchsia-checkout.py, unless the --skip-prepare option is used.
Invoke fx clean unless --skip-clean is used.
Setup the Bazel workspace, then run a series of tests to verify that things work properly.
Always try to add new tests when introducing new features under //build/bazel/, and run this script when changing its implementation (for now this is all manual, but will likely be automated in CQ in the future).
To help debug hermeticity issues that happen within repository rules, the Bazel workspace is configured to save a log of repository-related events to a file located under $BAZEL_TOPDIR/logs/workspace-events.log, this is a binary proto file (see https://bazel.build/remote/workspace) that can be converted to text automatically with //build/bazel/scripts/parse-workspace-event-log.py which can be invoked directly from your Fuchsia directory.
The script will automatically find the log for the latest command and dump its content to text on stdout, or to a file if the --output=FILE option is used.
The log files are rotated, up to 3 older revisions are stored in the $BAZEL_TOPDIR/logs directory so you can compare them if possible.