Continuous integration guide for Fuchsia

This is a guide to using LUCI, Chromium's build system, to continuously build and test Fuchsia projects.

We're going to do this in two steps:

  1. Configure scheduled builds. Tell LUCI how to build your project and set up a schedule for continuous integration.
  2. After this is done, you can set up a commit queue to block commits in Gerrit that would break the build.

If you‘ve already done part of this work, or if it’s set up already and you just need to change the configuration, this guide should still help you understand the LUCI concepts and how Fuchsia uses them.

This repo also contains example configuration files that you can copy into your project and alter as needed.

Step one: Set up scheduled builds

A branch named infra/config is the first place LUCI will look for configuration. Copy the example files from this directory to get started. The following sections will go through the individual files.

project.cfg on the infra/config branch

This is the top-level configuration file for a project. In LUCI, project is a high-level organizing concept. In Fuchsia, one project usually corresponds to one repo.

cr-buildbucket.cfg on the infra/config branch

cr-buildbucket.cfg tells LUCI how to build the project using Buildbucket.

Buildbucket is LUCI's build queue. It defines operations for:

  • Scheduling a build job.
  • Leasing a build in order to execute it.
  • Marking a build as successful or failed.

Buildbucket is a generic build queue, which means it's just a set of API definitions, and we need a service that actually implements it. We use Swarming as our implementation. Swarming is a big Chromium project that serves many purposes, but you can think of it as the thing that actually runs the Fuchsia build jobs.

You probably want to keep the ACL section as-is. In the example file, the project-fuchsia-tryjob-access group can schedule builds, and anyone can see them.

In the Swarming secion of the cr-buildbucket.cfg, we specify the hostname of our Swarming instance (always the main Chromium instance at chromium-swarm.appspot.com) and a set of builders.

In Buildbucket terminology, a builder is a specification for a machine that can carry out a specific build task. You'll want a builder for each target platform, and possibly separate debug and release builders. The example file defines two builders: x86 and ARM.

Each builder specifcation includes a category field, to group related builds.

The actual task that the builder carries out is called a recipe. A recipe is basically just a sequence of commands to run on the builder machine, specified in Python. The recipe framework has a user guide if you want to learn more about it, but most Fuchsia projects share the same build recipe, which is defined in the “infra/recipes” repo.

Recipes are configured in two parts of cr-buildbucket.cfg. First, in the swarming field, there is a builder_defaults field which contains recipe settings common to all builders. In the example file, this is where we say to use the standard Fuchsia recipe, and we give it a modules property to pass in, along with some other standard properties like the URL of the Fuchsia manifest repo. module should refer to the name of a config file in the packages repo, which will build a particular Fuchsia package.

luci-logdog.cfg on the infra/config branch

LogDog is the part of LUCI that collects logs from the recipes when they run. A luci-logdog.cfg file is required alongside cr-buildbucket.cfg to tell LUCI where to put the logs and what permissions to use. We generally use the same LogDog settings across Fuchsia projects, so you don't need to change anything.

luci-scheduler.cfg on the infra/config branch

This file tells LUCI which builds to run continuously. It consists of multiple job declarations, each one of which refers to a builder using the names defined in cr-buildbucket.cfg. This is basically cron for LUCI, and you can use cron expressions in the schedule parameter of each job. By convention in Fuchsia, we use “with 10m interval” as the schedule for our continuous jobs.

Global LUCI configuration

The last thing you need to do before you test your configuration is make sure that LUCI knows about your repo in its global configuration. Currently we share a LUCI instance with the Chrome team, and this configuration is in their Google-internal infrastructure repo. Ask someone from the Fuchsia infrastructure team to get your project added.

Testing your configuration

Right now the only real way to test your LUCI configuration is to commit a change and see if it works, then commit a fix if it doesn't. Once your configuration gets committed, your scheduled builds should start showing up on the LUCI Scheduler web interface.

However, you can test your recipe locally, by looking at the recipe properties in cr-buildbucket.cfg and passing them as JSON to the --properties argument of the referenced recipes.py script. In the example cr-buildbucket.cfg, the recipe is configured like this:

recipe {
  name: "fuchsia"
  repository: "https://fuchsia.googlesource.com/infra/recipes"
  properties: "remote:https://fuchsia.googlesource.com/manifest"
  properties: "manifest:userspace"
  properties_j: "modules:[\"example\"]"
}

This says that the recipe can be found in the “infra/recipes” repo, with the name “fuchsia”, and that it should be always be run with a certain set of properties. (This is a little confusing because one of those properties is the URL of a different repo, the Fuchsia manifest. Note that these URLs are used by different systems at different times: repository is used by Buildbucket to find the recipe. remote is used by the Fuchsia recipe to find the Fuchsia manifest.)

There are also additional properties for each specific builder. For example, the x86 builder includes this:

recipe {
  properties: "build_type:debug"
  properties: "target:x86-64"
}

So, to test the x86 build, you can put all of these properties together in a temporary file called example.json, like this:

{
  "remote": "https://fuchsia.googlesource.com/manifest",
  "manifest": "userspace",
  "modules": ["example"],
  "build_type": "debug",
  "target": "x86-64"
}

And then run the recipes.py script from the root directory of infra/recipes, which is the place we told Buildbucket to find our recipe:

python recipes.py run --properties-file example.json fuchsia

If that correctly builds the part of Fuchsia you want to build, then your recipe is probably OK. However, you'll still need to actually commit to make sure your configuration is valid.

Step two: Set up the commit queue

refs.cfg on the infra/config branch

This file tells LUCI where in the repo it should look for additional configuration files. In the case of a Fuchsia repo that uses the standard Fuchsia build recipe, the additional config file we care about is cq.cfg, described below.

By convention in Fuchsia we always use the path infra/config on the master branch. Note that this is different from the infra/config branch, which can be a bit confusing.

infra/config/cq.cfg on the master branch

This the configuration file for the Chromium Commit Queue, which is a thing that watches Gerrit for CLs it can build, test, and commit.

We always use the same cq_status_url for Fuchsia, and the git_repo_url is specific to the repo. cq_name is a unique identifier for the project's CQ, and it can usually just be the same as the all-lowercase name of the repo.

A LUCI commit queue configuration consists of one or more verifiers, which are steps that must pass before committing is allowed. The basic configuration for a Fuchsia CQ is a single try-job verifier, which means that every commit must pass a check that involves running a set of Builbucket jobs.

These jobs are specified in cr-buildbucket.cfg on the infra/config branch (see above) and referred to by their name field. You probably want to run all of your builders on the CQ. That means listing each one individually here. The example file includes the standard ARM/x86 builders.

Testing the commit queue

To test the commit queue, create a new commit for your project in Gerrit. The contents of the commit don't matter, since you will only be doing a dry run.

If you have the right permissions in Gerrit, you should see an option labeled “Commit-Queue” when you click on the “Reply” button, like so:

Gerrit screenshot

Select the “+1” option as shown above, which signals a dry run, and click “Send”. Very soon after that you should see a comment from the CQ bot, with a link to its progress. Check to see if it completes all the builds you configured in the above steps.