commit | 96a00cd0417ed7f8b8b73d766760292f83b8d066 | [log] [tgz] |
---|---|---|
author | Mark Seaborn <mseaborn@google.com> | Tue Apr 17 10:12:02 2018 -0700 |
committer | Mark Seaborn <mseaborn@google.com> | Tue Apr 17 10:12:02 2018 -0700 |
tree | f1bb60a3fc996ae157649dfac94f5509d9a59707 | |
parent | 3dc05a47968b36383a6d8950be940b5ce143b083 [diff] |
Add builder for running performance tests on hardware bots (Intel NUCs) Add "garnet-x64-perf-swift_canyon" builder. Change-Id: I62d66a3276e5e6e0c48f84a3ab63e63dd9d3f5c0
This repository contains Fuchsia's configuration files for LUCI, the continuous integration system that we share with the Chromium project.
You can configure LUCI to:
The configuration repos for Fuchsia continuous integration are included in the infra
Jiri manifest. To get the code:
jiri import infra https://fuchsia.googlesource.com/manifest jiri update
Your top-level Fuchsia directory will now have an infra
directory, containing the config
and recipes
repos, each of which is described in more detail below.
Buildbucket is LUCI's build queue. We use Buildbucket with Swarming, which is a system for centrally scheduling jobs to run on different machines. Swarming is the same system that the Chromium project uses.
cr-buildbucket.cfg defines a set of builders for Fuchsia. A builder basically defines a task to run, and a type of machine to run it on.
Here's one of our builders, as an example:
builders { category: "Fuchsia" name: "fuchsia-x86_64-linux-debug" mixins: "fuchsia" dimensions: "os:Ubuntu" recipe { properties: "target:x64" properties: "build_type:debug" } }
This builder has:
There‘s also reference to a builder mixin, which is a way to reuse configuration between builders. In this case, we’re using the “fuchsia” mixin, which is defined like this:
builder_mixins { name: "fuchsia" recipe { name: "fuchsia" properties: "manifest:fuchsia" } }
This mixin provides additional information about the recipe: it says which recipe to use (“fuchsia”, which corresponds to a file called fuchsia.py in the recipes repo) and it gives an additional property: it tells the recipe to use the Jiri manifest named “fuchsia” when it checks out the code.
And finally, this builder is in a bucket, which is a grouping of builders with the same set of ACLs. We have two buckets:
This particular builder is in the continuous bucket, which declares a set of defaults for all of its builders:
builder_defaults { swarming_tags: "allow_milo:1" dimensions: "pool:Fuchsia-try" dimensions: "netboot_devices:0" recipe { repository: "https://fuchsia.googlesource.com/infra/recipes" properties: "remote:https://fuchsia.googlesource.com/manifest" } }
There are more machine dimensions here, including the machine pool, since the continuous jobs and try jobs use different pools. There's also the location of the recipes repos, and the “remote” property for the recipe, which points to the Fuchsia manifest repo.
Since you want to schedule continuous builds and a commit queue, you need to add the following:
luci-scheduler.cfg tells LUCI which builds to run continuously. It consists of multiple job
declarations, each one of which refers to a builder using the names defined in the Buildbucket configuration. This is basically cron for LUCI, and you can use cron expressions in the schedule
parameter of each job, as well as expressions like: “with 10m interval”.
Setting up a commit queue for a repo requires three steps:
Add a new directory to the repositories directory of the infra/config
repo. As the name implies repositories
contains a subdirectory for every repo that has CQ enabled. Your new directory should contain refs.cfg
and cq.cfg
, described in detail below. You can copy them from an existing directory to get started.
refs.cfg
This is a magical workaround for legacy issues. It's basically the file that bootstraps the commit queue configuration. Just make sure that config_path
points to your new directory.
cq.cfg
This the configuration file for the Chromium Commit Queue, which is a thing that watches Gerrit for CLs it can build, test, and commit.
cq_name
is a unique identifier for the project's CQ, and it can usually just be the same as the all-lowercase name of the repo. git_repo_url
is used to specify the repository.
A LUCI commit queue configuration consists of one or more verifiers, which are steps that must pass before committing is allowed. The basic configuration for a Fuchsia CQ is a single try-job verifier, which means that every commit must pass a check that involves running a set of builders from the Buildbucket configuration.
Under buckets
, we always specify that we are using “luci.fuchsia.try”. Add an entry here for every builder in the try bucket that you want to run on the CQ for this repo. This could include builders specific to your repo, as well as anything else you don't want to break when you commit things, such as the main Fuchsia build.
The gerrit_cq_ability
section defines permissions for who can use the CQ, and it's the same for all Fuchsia repos.
To do this, open a JIRA ticket in the INTK project, asking to enable the Chromium CQ bot for your repo.
We use the Chromium CQ bot, which is a service provided by Chromium infrastructure. There are a couple of Google-internal repos that require changes for every new Fuchsia repo with CQ, which the Infra team can do for you.
Once everything else is done, you'll need a Fuchsia Gerrit administrator to turn on all the appropriate CQ buttons in Gerrit. Open a JIRA ticket in the INTK project, asking to enable CQ for your repo in Gerrit (or just add a comment on your existing ticket from step 2).
If you are a Gerrit administrator, you can do it yourself. Go to the project administration page for the repo you want to change, and open the “Access” tab at the top (you‘ll need to temporarily disable PolyGerrit to get to this tab). Here’s an example link to the right page, for the test_runner
repo: https://fuchsia-review.googlesource.com/?polygerrit=0#/admin/projects/test_runner,access
Click the “Edit” button and change the text in the “Rights Inherit From” field from “All-Projects” to “Commit-Queue”. Then click “Save Changes”. Now the CQ UI elements should show up on any CLs on the repo.
Right now the only real way to test your LUCI configuration is to commit a change and see if it works, then commit a fix if it doesn't. Once your configuration gets committed, your scheduled builds should start showing up on the LUCI Scheduler web interface.
To test the commit queue, create a new commit for your project in Gerrit. The contents of the commit don't matter, since you will only be doing a dry run.
If you have the right permissions in Gerrit and you are using PolyGerrit, you should see a button in the top right that says “CQ Dry Run”. Click it, and soon you should see a comment from the CQ bot, with a link to its progress. Check to see if it completes all the builds you configured in the above steps.