DEPS: gerrit, git, recipe_engine/context, recipe_engine/json, recipe_engine/path, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/step, recipe_engine/time, recipe_engine/url
API for writing auto-roller recipes.
— def attempt_roll(self, gerrit_project, repo_dir, commit_message, commit_untracked=False, dry_run=False):
Attempts to submit local edits via the CQ.
It additionally has two modes of operation, dry-run mode and production mode. The precise steps it performs are as follows:
It assumes that repo_dir contains unstaged changes to only tracked files.
Args: gerrit_project (str): The name of the project to roll to in Gerrit, which is local to current Gerrit host as defined by api.gerrit.host(). For example, “garnet” would be a valid gerrit_project for fuchsia-review.googlesource.com. repo_dir (Path): The path to the directory containing a local copy of the git repo with changes that will be rolled. commit_message (str): The commit message for the roll. Note that this method will automatically append a Gerrit Change ID to the change. Also, it may be a multiline string (embedded newlines are allowed). commit_untracked (bool): Whether to commit untracked files as well. dry_run (bool): Whether to execute this method in dry_run mode.
@property
— def poll_interval_secs(self):
Returns how many seconds roll() will wait in between each poll.
Defined by the input property with the same name.
@property
— def poll_timeout_secs(self):
Returns how many seconds roll() will poll for.
Defined by the input property with the same name.
DEPS: recipe_engine/cipd, recipe_engine/context, recipe_engine/path, recipe_engine/step
Provides steps to run bazel.
— def ensure_bazel(self, version=‘latest’):
Ensures that bazel is installed, returning its path.
Additional calls with the same version are no-ops. Additional calls with a different version are errors.
Args: version: The CIPD version to install. Returns: The Path to the bazel binary. Raises: AssertionError: if this method has already been called with a different version.
DEPS: recipe_engine/cipd, recipe_engine/context, recipe_engine/path, recipe_engine/raw_io, recipe_engine/step
APIs for retrieving the build ID for a given buildset and builder ID.
— def __call__(self, step_name, builder, buildset, leak_to=None):
Retrieves the build ID for a given buildset and builder ID.
Args: step_name (str): The name of the step produced. builder (str): A fully-qualified buildbucket v2 builder ID, consisting of //. For example: ‘fuchsia/ci/garnet-x64-release-qemu_kvm’. buildset (str): A fully-qualified buildbucket V2 buildset tag, consisting of commit/gitiles///+/. For example: ‘commit/gitiles/fuchsia.googlesource.com/topaz/+/e3127e0bd6d57da7a5959ee70eb0a396590e6d53’. leak_to (Path): If leak_to is provided, it must be a Path object. This path will be used in place of a random temporary file, and the file will not be deleted at the end of the step.
@property
— def buildset_lookup_tool(self):
— def ensure_buildset_lookup(self, version=None):
Ensures that the buildset lookup tool is installed.
DEPS: recipe_engine/cipd, recipe_engine/context, recipe_engine/path, recipe_engine/step
CatapultApi provides support for the Catapult infra tool.
— def __call__(self, *args, **kwargs):
Return a catapult command step.
— def ensure_catapult(self, version=None):
— def upload(self, input_file, url, timeout=None, **kwargs):
Uploads performance JSON data to a dashboard.
Args: input_file (Path): Full path to the input file to upload. url (string): The url to upload data to. timeout (string): Optional request timeout duration string. e.g. 12s or 1m. kwargs: Keyword argments passed to the returned step.
Returns: A step to execute the upload subcommand.
DEPS: gerrit, jiri, recipe_engine/buildbucket, recipe_engine/context, recipe_engine/json, recipe_engine/path, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/source_manifest, recipe_engine/step
An abstraction over how Jiri checkouts are created during Fuchsia CI/CQ builds.
— def __call__(self, manifest, remote, project=None, build_input=None, timeout_secs=None, run_hooks=True, override=False):
Initializes and populates a jiri checkout from a remote manifest.
Emits a source manifest for the build.
Args: manifest (str): Relative path to the manifest in the remote repository. remote (str): URL to the remote repository. project (str): The name that jiri should assign to the project. build_input (buildbucket.build_pb2.Build.Input): The input to a buildbucket build. timeout_secs (int): A timeout for jiri update in seconds. run_hooks (bool): Whether or not to run the hooks. override (bool): Whether to override the imported manifest with a commit's given revision.
— def from_commit(self, manifest, remote, commit, project, run_hooks, override, timeout_secs):
Initializes and populates a Jiri checkout from a remote manifest and Gitiles commmit.
Args: manifest (str): Relative path to the manifest in the remote repository. remote (str): URL to the remote repository. project (str): The name that jiri should assign to the project. remote (str): The remote git repository. commit: Commit information derived from buildbucket.build_pb2.Build.Input.gitiles_commit. timeout_secs (int): A timeout for jiri update in seconds. run_hooks (bool): Whether or not to run the hooks. override (bool): Whether to override the imported manifest with a commit's given revision.
— def from_patchset(self, manifest, remote, project, run_hooks, timeout_secs, gerrit_change):
Initializes and populates a Jiri checkout from a remote manifest and Gerrit change.
Args: manifest (str): Relative path to the manifest in the remote repository. remote (str): URL to the remote repository. project (str): The name that jiri should assign to the project. build_input (buildbucket.build_pb2.Build.Input): The input to a buildbucket build. timeout_secs (int): A timeout for jiri update in seconds. gerrit_change: An element from buildbucket.build_pb2.Build.Input.gerrit_changes.
— def from_snapshot(self, snapshot, timeout_secs=None):
Initializes and populates a jiri checkout from a snapshot.
Additionally emits a source manifest for the build.
Args: snapshot (Path): Path to the jiri snapshot. timeout_secs (int): A timeout for jiri update in seconds.
DEPS: recipe_engine/cipd, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/python, recipe_engine/raw_io, recipe_engine/step
ClangTidyApi provides support for ClangTidy.
— def ensure_clang(self):
— def gen_compile_commands(self, checkout_dir):
— def get_line_from_offset(self, f, offset):
Get the file line and char number from a file offset.
Clang-Tidy emits warnings that mark the location of the error by the char offset from the beginning of the file. This converts that number into a line and char position.
Args: filename (str): Path to file. offset (int): Offset to convert.
— def run(self, step_name, filename, compile_commands, checks=[‘*’]):
Runs clang-tidy on the specified file and returns parsed json output.
TODO(juliehockett): Consider packaging and using the run_clang_tidy script.
Args: filename (str): Path to file on which clang-tidy should be run. compile_commands (str): Path to dir containing compile_commands.json. checks (List): List of checks to run for clang-tidy (default is whatever is specified in the .clang-tidy file).
DEPS: recipe_engine/cipd, recipe_engine/context, recipe_engine/path, recipe_engine/raw_io, recipe_engine/step
Module for interacting with CloudKMS.
This is a thin wrapper of the CloudKMS Go client at https://github.com/luci/luci-go/client/cmd/cloudkms
— def decrypt(self, step_name, crypto_key_path, ciphertext_file, plaintext_file):
Decrypts a ciphertext encrypted with a CloudKMS crypto key.
Args: step_name (str): name of the step. crypto_key_path (str): path in CloudKMS to the crypto key, generically of the form <project>/<location>/<key ring>/<crypto key name>, where the infixes are CloudKMS concepts detailed at https://cloud.google.com/kms/docs/object-hierarchy. ciphertext_file (Path): path to a file containing the ciphertext. plaintext_file (Path): path to a file to which the plaintext will be written.
Returns: A step to perform the decryption.
— def ensure_cloudkms(self, version=None):
DEPS: checkout, cloudkms, gerrit, git, gn, goma, gsutil, isolated, jiri, minfs, ninja, qemu, swarming, tar, testsharder, zbi, recipe_engine/buildbucket, recipe_engine/cipd, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/raw_io, recipe_engine/step
APIs for checking out, building, and testing Fuchsia.
— def analyze_collect_result(self, step_name, result, build_dir, symbolizer_output=None):
Analyzes a swarming.CollectResult and reports results as a step.
Args: step_name (str): The display name of the step for this analysis. result (swarming.CollectResult): The swarming collection result to analyze. build_dir (Path): A path to the build directory for symbolization artifacts. symbolizer_output (Path|None): A path to a file to write symbolized output.
Raises: A StepFailure if a kernel panic is detected, or if the tests timed out. An InfraFailure if the swarming task failed for a different reason.
— def build(self, target, build_type, packages, variants=(), gn_args=[], ninja_targets=(), board=None, product=None, zircon_args=[], collect_build_metrics=False, build_for_testing=False, build_archive=False, build_package_archive=False):
Builds Fuchsia from a Jiri checkout.
Expects a Fuchsia Jiri checkout at api.path[‘start_dir’].
Args: target (str): The build target, see TARGETS for allowed targets build_type (str): One of the build types in BUILD_TYPES packages (sequence[str]): A sequence of packages to pass to GN to build variants (sequence[str]): A sequence of build variant selectors to pass to GN in select_variant gn_args (sequence[str]): Additional arguments to pass to GN ninja_targets (sequence[str]): Additional target args to pass to ninja board (str): A board to pass to GN to build product (str): A product to pass to GN to build zircon_args (sequence[str]): A sequence of Make arguments to pass when building zircon. collect_build_metrics (bool): Whether to collect build metrics. build_for_testing (bool): Whether to build the basic images needed to boot and test on fuchsia. build_archive (bool): Whether to build an image archive to be uploaded. build_package_archive (bool): Whether to build a package archive to be uploaded, to be used for updating.
Returns: A FuchsiaBuildResults, representing the recently completed build.
— def checkout(self, build, manifest, remote, project=None, timeout_secs=(40 * 60)):
Uses Jiri to check out a Fuchsia project.
The root of the checkout is returned via FuchsiaCheckoutResults.root_dir.
Args: build (buildbucket.build_pb2.Build): A buildbucket build. manifest (str): A path to the manifest in the remote (e.g. manifest/minimal) remote (str): A URL to the remote repository which Jiri will be pointed at project (str): The name of the project timeout_secs (int): How long to wait for the checkout to complete before failing
Returns: A FuchsiaCheckoutResults containing details of the checkout.
— def checkout_patched_snapshot(self, gerrit_change, timeout_secs=(20 * 60)):
Uses Jiri to check out Fuchsia from a Jiri snapshot from a Gerrit patch. The root of the checkout is returned via FuchsiaCheckoutResults.root_dir.
Args: gerrit_change (buildbucket.common_pb2.GerritChange): A Gerrit change. timeout_secs (int): How long to wait for the checkout to complete before failing
Returns: A FuchsiaCheckoutResults containing details of the checkout.
— def checkout_snapshot(self, repository=None, revision=None, gitiles_commit=None, timeout_secs=(20 * 60)):
Uses Jiri to check out Fuchsia from a Jiri snapshot.
The root of the checkout is returned via FuchsiaCheckoutResults.root_dir.
Args: repository (str): A URL to the remote repository containing the snapshot. The snapshot should be available at the top-level in a file called ‘snapshot’. revision (str): The git revision to check out from the repository. gitiles_commit (buildbucket.common_pb2.GitilesCommit): A Gitiles commit. timeout_secs (int): How long to wait for the checkout to complete before failing.
Returns: A FuchsiaCheckoutResults containing details of the checkout.
@property
— def results_dir_on_host(self):
The directory on host to which host and target test results will be written.
Target test results will be copied over to this location and host test results will be written here. Host and target tests on should write to separate subdirectories so as not to collide.
@property
— def results_dir_on_target(self):
The directory on target to which target test results will be written.
— def test(self, build, test_pool, test_cmds, device_type, timeout_secs=(40 * 60), pave=True, external_network=False, requires_secrets=False, extra_dimensions={}, host_cmd=[]):
Tests a Fuchsia build on the specified device.
Expects the build and artifacts to be at the same place they were at the end of the build.
Args: build (FuchsiaBuildResults): The Fuchsia build to test. test_pool (str): Swarming pool from which the test task will be drawn. timeout_secs (int): The amount of seconds to wait for the tests to execute before giving up. external_network (bool): Whether to enable access to the external network when executing tests. Ignored if device_type != ‘QEMU’. requires_secrets (bool): Whether tests require plaintext secrets; ignored if device_type == ‘QEMU’. pave (bool): Whether to pave the image to disk. Ignored if device_type == ‘QEMU’. extra_dimensions (dict): Additional dimensions for swarming allocation host_cmd (List(string)): Command to run on host after paving
Returns: A FuchsiaTestResults representing the completed test.
— def test_in_shards(self, test_pool, build, environment_label='', timeout_secs=(40 * 60)):
Tests a Fuchsia build by sharding.
Expects the build and artifacts to be at the same place they were at the end of the build.
Args: test_pool (str): The Swarming pool to schedule test tasks in. build (FuchsiaBuildResults): The Fuchsia build to test. environment_label (str): test environment label on which the testsharder will key. timeout_secs (int): The amount of seconds to wait for the tests to execute before giving up.
Returns: A list of FuchsiaTestResults representing the completed test tasks that were not subject to an infra failure.
— def test_on_host(self, build):
Tests a Fuchsia build from the host machine.
Args: build (FuchsiaBuildResults): The Fuchsia build to test.
Returns: A FuchsiaTestResults representing the completed test.
DEPS: recipe_engine/cipd, recipe_engine/context, recipe_engine/json, recipe_engine/path, recipe_engine/step
Module for querying a Gerrit host through the Gerrit API.
— def abandon(self, name, change_id, message=None, test_data=None):
Abandons a change.
Returns the details of the change, after attempting to abandon.
Args: name (str): The name of the step. change_id (str): A change ID that uniquely defines a change on the host. message (str): A message explaining the reason for abandoning the change. test_data (recipe_test_api.StepTestData): Test JSON output data for this step.
— def change_details(self, name, change_id, gerrit_host=None, query_params=[], test_data=None):
Returns a JSON dict of details regarding a specific change.
Args: name (str): The name of the step. change_id (str): A change ID that uniquely defines a change on the host. gerrit_host (str): The Gerrit host to make the query against. Overrides the recipe module's global host property. query_params (list): A list of Gerrit REST query parameters (strings). Documented at https://gerrit-review.googlesource.com/Documentation/rest-api-changes.html#query-options test_data (recipe_test_api.StepTestData): Test JSON output data for this step.
— def create_change(self, name, project, subject, branch, topic=None, test_data=None):
Creates a new change for a given project on the gerrit host.
Returns the details of the newly-created change.
Args: name (str): The name of the step. project (str): The name of the project on the host to create a change for. subject (str): The subject of the new change. branch (str): The branch onto which the change will be made. topic (str): A gerrit topic that can be used to atomically land the change with other changes in the same topic. test_data (recipe_test_api.StepTestData): Test JSON output data for this step.
— def ensure_gerrit(self, version=None):
@host.setter
— def host(self, host):
— def restore_change(self, name, change_id, message=None, test_data=None):
Restores a change.
Returns the details of the change, after attempting to restore.
Args: name (str): The name of the step. change_id (str): A change ID that uniquely defines a change on the host. message (str): An optional message explaining the reason for restoring the change. test_data (recipe_test_api.StepTestData): Test JSON output data for this step.
— def set_review(self, name, change_id, labels=None, reviewers=None, ccs=None, revision=‘current’, test_data=None):
Sets a change at a revision for review. Can optionally set labels, reviewers, and CCs.
Returns updated labels, reviewers, and whether the change is ready for review as a JSON dict.
Args: name (str): The name of the step. change_id (str): A change ID that uniquely defines a change on the host. labels (dict): A map of labels (with names as strings, e.g. ‘Code-Review’) to the integral values you wish to set them to. reviewers (list): A list of strings containing reviewer IDs (e.g. email addresses). ccs (list): A list of strings containing reviewer IDs (e.g. email addresses). revision (str): A revision ID that identifies a revision for the change (default is ‘current’). test_data (recipe_test_api.StepTestData): Test JSON output data for this step.
DEPS: recipe_engine/context, recipe_engine/file, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/raw_io, recipe_engine/step
GitApi provides support for Git.
— def __call__(self, *args, **kwargs):
Return a git command step.
— def checkout(self, url, path=None, ref=None, recursive=False, submodules=True, submodule_force=False, remote=‘origin’, file=None, **kwargs):
Checkout a given ref and return the checked out revision.
Args: url (str): url of remote repo to use as upstream path (Path): directory to clone into ref (str): ref to fetch and check out recursive (bool): whether to recursively fetch submodules or not submodules (bool): whether to sync and update submodules or not submodule_force (bool): whether to update submodules with --force remote (str): name of the remote to use file (str): optional path to a single file to checkout
— def commit(self, message, files=(), all_tracked=False, all_files=False, **kwargs):
Runs git commit in the current working directory. Args: message (str): The message to attach to the commit. files (seq[Path]): The set of files containing changes to commit. all_tracked (bool): Stage all tracked files before committing. If True, files must be empty and all_files must be False. all_files (bool): Stage all files (even untracked) before committing. If True, files must be empty and all_tracked must be False.
— def get_hash(self, commit=‘HEAD’, **kwargs):
Find and return the hash of the given commit.
— def get_timestamp(self, commit=‘HEAD’, test_data=None, **kwargs):
Find and return the timestamp of the given commit.
— def push(self, ref, remote=‘origin’, **kwargs):
— def rebase(self, branch=‘master’, remote=‘origin’, **kwargs):
Run rebase HEAD onto branch
DEPS: recipe_engine/cipd, recipe_engine/context, recipe_engine/json, recipe_engine/path, recipe_engine/python, recipe_engine/raw_io, recipe_engine/step, recipe_engine/url
Module for polling a Git repository using the Gitiles web interface.
— def ensure_gitiles(self, version=None):
— def fetch(self, url, file_path, branch=‘master’, step_name=None, timeout=None, test_data=None):
Downloads raw file content from a Gitiles repository.
Args: url (str): base URL to the repository. file_path (str): relative path to the file from the repository root. branch (str): branch of the repository. step_name (str): custom name for this step (optional). timeout (int): number of seconds to wait before failing.
Returns: Raw file content.
— def log(self, url, treeish, limit=0, step_name=None, test_data={}):
Returns the most recent commits for treeish object.
Args: url (str): base URL of the remote repository. treeish (str): tree object identifier. limit (int): number of commits to limit the fetching to. step_name (str): custom name for this step (optional).
— def refs(self, url, refspath=‘refs’, step_name=‘refs’, test_data=[]):
Resolves each ref in a repository to git revision
Args: url (str): URL of the remote repository. refspath (str): limits which refs to resolve.
DEPS: recipe_engine/path, recipe_engine/step
APIs for interacting with build systems.
Clients must call set_path before using this module, to point to the prebuilt GN binary.
— def set_path(self, path):
Sets the path to the GN executable.
DEPS: recipe_engine/cipd, recipe_engine/context, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/python, recipe_engine/raw_io, recipe_engine/step
GoApi provides support for Go.
— def __call__(self, *args, **kwargs):
Return a Go command step.
— def ensure_go(self, version=None):
Ensures that go distribution is installed.
@property
— def go_executable(self):
@property
— def go_root(self):
— def inline(self, program, add_go_log=True, **kwargs):
Run an inline Go program as a step. Program is output to a temp file and run when this step executes.
DEPS: recipe_engine/cipd, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/raw_io, recipe_engine/step
GomaApi contains helper functions for using goma.
@contextmanager
— def build_with_goma(self, env={}):
Make context wrapping goma start/stop.
Raises: StepFailure or InfraFailure if it fails to build.
— def ensure_goma(self, canary=False):
@property
— def goma_ctl(self):
@property
— def goma_dir(self):
@property
— def jobs(self):
Returns number of jobs for parallel build using Goma.
Uses value from property “$infra/goma:{“jobs”: JOBS}” if configured (typically in cr-buildbucket.cfg), else defaults to recommended_goma_jobs.
@property
— def json_path(self):
@property
— def jsonstatus(self):
@property
— def recommended_goma_jobs(self):
Return the recommended number of jobs for parallel build using Goma.
Prefer to use just goma.jobs and configure it through default builder properties in cr-buildbucket.cfg.
This function caches the _recommended_jobs.
— def start(self, env={}, **kwargs):
Start goma compiler_proxy.
A user MUST execute ensure_goma beforehand. It is user's responsibility to handle failure of starting compiler_proxy.
— def stop(self, **kwargs):
Stop goma compiler_proxy.
A user MUST execute start beforehand. It is user's responsibility to handle failure of stopping compiler_proxy.
Raises: StepFailure if it fails to stop goma.
DEPS: recipe_engine/cipd, recipe_engine/context, recipe_engine/json, recipe_engine/path, recipe_engine/python, recipe_engine/step
GSUtilApi provides support for GSUtil.
— def __call__(self, *args, **kwargs):
Return a step to run arbitrary gsutil command.
— def copy(self, src_bucket, src, dst_bucket, dst, link_name=‘gsutil.copy’, unauthenticated_url=False, **kwargs):
— def ensure_gsutil(self, version=None):
@non_step
— def join(self, *parts):
Constructs a GS path from composite parts.
@non_step
— def normalize(self, url):
Normalize a GS URL using the gs:// URL prefix.
— def upload(self, bucket, src, dst, link_name=‘gsutil.upload’, unauthenticated_url=False, **kwargs):
DEPS: recipe_engine/path, recipe_engine/python, recipe_engine/raw_io
HashApi provides file hashing functionality.
— def md5(self, name, source, test_data=''):
— def sha1(self, name, source, test_data=''):
— def sha224(self, name, source, test_data=''):
— def sha256(self, name, source, test_data=''):
— def sha384(self, name, source, test_data=''):
— def sha512(self, name, source, test_data=''):
DEPS: recipe_engine/cipd, recipe_engine/context, recipe_engine/json, recipe_engine/path, recipe_engine/raw_io, recipe_engine/step
APIs for interacting with isolates.
— def __call__(self, *args, **kwargs):
Return an isolate command step.
— def ensure_isolated(self, version=None):
Ensures that isolate client is installed.
@isolate_server.setter
— def isolate_server(self, value):
Changes URL of Isolate server to use.
— def isolated(self):
Returns an Isolated object that can be used to archive a set of files and directories.
@property
— def isolated_client(self):
DEPS: gerrit, recipe_engine/cipd, recipe_engine/context, recipe_engine/json, recipe_engine/path, recipe_engine/raw_io, recipe_engine/source_manifest, recipe_engine/step
JiriApi provides support for Jiri managed checkouts.
— def __call__(self, *args, **kwargs):
Return a jiri command step.
— def clean(self, all=False, **kwargs):
— def edit_manifest(self, manifest, projects=None, imports=None, test_data=None, **kwargs):
Creates a step to edit a Jiri manifest.
Args: manifest (str): Path to the manifest to edit relative to the project root. e.g. “manifest/zircon” projects (List): A heterogeneous list whose entries are either: * A string representing the name of a project to edit. * A (name, revision) tuple representing a project to edit. imports (List): The same as projects, except each list entry represents an import to edit.
Returns: A step to edit the manifest.
— def emit_source_manifest(self):
Emits a source manifest for this build for the current jiri checkout.
— def ensure_jiri(self, version=None):
— def import_manifest(self, manifest, remote, name=None, revision=None, overwrite=False, remote_branch=None, **kwargs):
Imports manifest into Jiri project.
Args: manifest (str): A file within the repository to use. remote (str): A remote manifest repository address. name (str): The name of the remote manifest project. revision (str): A revision to checkout for the remote. remote_branch (str): A branch of the remote manifest repository to checkout. If a revision is specified, this value is ignored.
Returns: A step result.
— def init(self, dir=None, **kwargs):
@property
— def jiri(self):
— def override(self, project, remote, new_revision=‘HEAD’):
Overrides a given project entry with a new revision.
Args: project (str): name of the project. remote (str): URL to the remote repository. new_revision (str|None): new revision to override the project's current.
— def patch(self, ref, host=None, project=None, delete=False, force=False, rebase=False, cherrypick=False):
— def project(self, projects=[], out=None, test_data=None):
Args: projects (List): A heterogeneous list of strings representing the name of projects to list info about (defaults to all). out (str): Path to write json results to, with the following schema: [ { “name”: “zircon”, “path”: “local/path/to/zircon”, “relativePath”: “zircon”, “remote”: “https://fuchsia.googlesource.com/zircon”, “revision”: “af8fd6138748bc11d31a5bde3303cdc19c7e04e9”, “current_branch”: “master”, “branches”: [ “master” ] } ... ]
Returns: A step to provide structured info on existing projects and branches.
— def read_manifest_element(self, manifest, element_type, element_name):
Reads information about a or from a manifest file.
Args: manifest (str|Path): Path to the manifest file. element_type (str): One of ‘import’ or ‘project’. element_name (str): The name of the element.
Returns: A dict containing the project fields. Any fields that are missing or have empty values are omitted from the dict. Examples:
# Read remote attribute of the returned project
print(project['remote']) # https://fuchsia.googlesource.com/my_project
# Check whether remote attribute was present and non-empty.
if 'remote' in project:
...
— def run_hooks(self, local_manifest=False, attempts=3):
— def snapshot(self, file=None, test_data=None, **kwargs):
— def source_manifest(self, file=None, test_data=None, **kwargs):
Generates a source manifest JSON file.
Args: file (Path): Optional output path for the source manifest.
Returns: The contents of the source manifest as a Python dictionary.
— def update(self, gc=False, rebase_tracked=False, local_manifest=False, run_hooks=True, snapshot=None, attempts=3, **kwargs):
DEPS: recipe_engine/cipd, recipe_engine/context, recipe_engine/path, recipe_engine/step
APIs for deriving the last-known-good-snapshot for a builder.
— def __call__(self, step_name, builder, output_file):
Retrieves the last-known-“good”-(jiri-)snapshot given a builder.
Args: step_name (str): The name of the step produced. builder (seq[str]): A list of fully-qualified buildbucket v2 builder ID, consisting of //. For example: [‘fuchsia/ci/garnet-x64-release-qemu_kvm’]. output_file (Path|Placeholder): The location to dump the retrieved snapshot.
— def ensure_lkgs(self, version=None):
Ensures that the lkgs tool is installed.
@property
— def lkgs_tool(self):
DEPS: recipe_engine/file, recipe_engine/path, recipe_engine/raw_io, recipe_engine/step
MinfsApi provides support for Fuchia's MinFS tool.
Currently this module can only be used with a Zircon build, which produces the local minfs binary.
— def copy_image(self, step_name, image_path, out_dir):
Copies everything from a minfs image into a local directory.
Args: step_name (str): The name of the step under which to copy the minfs files. image_path (Path): Full path to the minfs image. out_dir (Path): Full path to the directory to copy files into.
Returns: A step to perform the copy.
— def cp(self, from_path, to_path, image, **kwargs):
Copies a file or directory from an image.
Paths inside of the MinFS image are prefixed with ‘::’, so ‘::’ refers to the root of the MinFS image.
Args: from_path (str|Path): The path to copy from. to_path (str|Path): The path to copy to. image (str|Path): The path to the MinFS image.
Returns: A step to perform the copy.
— def create(self, path, size=‘100M’, **kwargs):
Creates a MinFS image at the given path.
Args: path (str): The path at which to create the image. size (str): The size of the image, number followed by unit. Defaults to 100M.
Returns: A step to perform the creation.
@minfs_path.setter
— def minfs_path(self, path):
The path to the minfs command.
DEPS: recipe_engine/json, recipe_engine/python, recipe_engine/raw_io
APIs for interacting with Ninja.
Clients must call set_path before using this module, to point to the prebuilt Ninja binary.
— def __call__(self, build_dir, targets, job_count=None, build_file=None, fail_threshold=None, verbose=False):
Runs Ninja.
Args: build_dir (Path): CD into this directory before executing. job_count (int): No. parallel jobs (Ninja default is to guess from available CPUs). targets (List(string)): List of targets to build. build_file (Path): Ninja build file (default=build.ninja) fail_threshold (int): Keep going until this many jobs fail (Ninja default of 1). verbose (bool): Show all command lines when building.
Returns: The step result of running Ninja.
— def set_path(self, path):
Sets the path to the Ninja executable.
DEPS: recipe_engine/cipd, recipe_engine/context, recipe_engine/path, recipe_engine/step
QemuApi provides support for QEMU.
— def create_image(self, image, backing_file, fmt=‘qcow2’):
Creates a QEMU image from a backing file.
Args: image (Path): Path to the image to be created. backing_file (Path): The backing file to use for the image. fmt (str): The format of the image.
— def ensure_qemu(self, version=None):
@property
— def qemu_img(self):
DEPS: isolated, recipe_engine/cipd, recipe_engine/context, recipe_engine/json, recipe_engine/path, recipe_engine/raw_io, recipe_engine/step
APIs for interacting with swarming.
— def __call__(self, *args, **kwargs):
Return a swarming command step.
— def collect(self, timeout=None, tasks_json=None, tasks=[]):
Waits on a set of Swarming tasks.
Returns both the step result as well as a set of neatly parsed results.
Args: timeout: timeout to wait for result. tasks_json: load details about the task(s) from the json file. tasks: list of task ids to wait on.
— def ensure_swarming(self, version=None):
Ensures that swarming client is installed.
— def spawn_tasks(self, tasks=(), json_output=None):
Spawns a set of Swarming tasks.
Args: tasks (seq[TaskRequest]): A sequence of task request objects representing the tasks we want to spawn. json_output (Path): Optional filepath to leak a JSON file containing the return value of this method.
Returns: A Python dict representing the JSON spawn response that may be passed into collect().
@property
— def swarming_client(self):
@swarming_server.setter
— def swarming_server(self, value):
Changes URL of Swarming server to use.
— def task_request(self, *args, **kwargs):
Creates a new TaskRequest object.
Passes down all arguments to the TaskRequest constructor with the exception of isolate_server, which is provided by the isolated recipe module.
— def trigger(self, name, raw_cmd, isolated=None, dump_json=None, dimensions=None, expiration=None, io_timeout=None, hard_timeout=None, idempotent=False, cipd_packages=None, outputs=None):
Triggers a Swarming task.
Args: name: name of the task. raw_cmd: task command. isolated: hash of isolated test on isolate server. dump_json: dump details about the triggered task(s). dimensions: dimensions to filter slaves on. expiration: seconds before this task request expires. io_timeout: seconds to allow the task to be silent. hard_timeout: seconds before swarming should kill the task. idempotent: whether this task is considered idempotent. cipd_packages: list of 3-tuples corresponding to CIPD packages needed for the task: (‘path’, ‘package_name’, ‘version’), defined as follows: path: Path relative to the Swarming root dir in which to install the package. package_name: Name of the package to install, eg. “infra/tools/authutil/${platform}” version: Version of the package, either a package instance ID, ref, or tag key/value pair. outputs: list of paths to files which can be downloaded via collect.
DEPS: recipe_engine/cipd, recipe_engine/context, recipe_engine/path, recipe_engine/platform, recipe_engine/python, recipe_engine/step
Provides steps to tar and untar files.
— def create(self, path, compression=None):
Returns TarArchive object that can be used to compress a set of files.
Args: path: path of the archive file to be created. compression: str, one of COMPRESSION_OPTS or None to disable compression.
— def ensure_tar(self, version=None):
Ensures that bsdtar is installed.
— def extract(self, step_name, path, directory=None, strip_components=None):
Uncompress |archive| file.
Args: step_name: name of the step. path: absolute path to archive file. directory: directory to extract the archive in. strip_components: strip number of leading components from file names.
DEPS: recipe_engine/cipd, recipe_engine/context, recipe_engine/json, recipe_engine/path, recipe_engine/step
Recipe module that wraps the testsharder tool, which searches a Fuchsia build for test specifications and groups them into shards.
The tool assigns a unique name to each produced shard.
Testsharder tool: https://fuchsia.googlesource.com/infra/infra/+/master/cmd/testsharder/
Module for interacting with the Testsharder tool.
The testsharder tool accepts a set of test specifications and produces a file containing shards of execution.
— def ensure_testsharder(self, version=‘latest’):
— def execute(self, step_name, fuchsia_build_dir, label='', output_file=None):
Executes the testsharder tool.
Args: step_name (str): name of the step. fuchsia_build_dir (Path): path to a Fuchsia build output directory for which GN has been run (ninja need not have been executed). label (str): a label on which to filter test specs. output_file (Path): optional file path to leak output to.
Returns: A list of Shards, each representing one test shard.
DEPS: recipe_engine/raw_io, recipe_engine/step
ZbiApi provides support for Zircon's ZBI tool.
— def copy_and_extend(self, step_name, input_image, output_image, manifest):
Creates a copy of a ZBI and extends its bootFS manifest.
A copy of |input_image| is made at |output_image|, and the files given in |manifest| are added to the latter's bootFS manifest.
Args: step_name (str): The name of the step. input_image (Path): The path to the input image. output_image (Path): The path to the output image. manifest (dict[str]Path): a dictionary of destination-to-source mappings, where destination/source are paths to files or directories on target/host, respectively. Note that sources cannot be placeholders.
Returns: A step to perform the operation.
— def copy_and_overwrite_cmdline(self, step_name, input_image, output_image, cmdline_file):
Creates a copy of a ZBI and overwrites its kernel command line.
A copy of |input_image| is made at |output_image|, and the file given as |cmdline_file| is written as the kernel command line.
Args: step_name (str): The name of the step. input_image (Path): The path to the input image. output_image (Path): The path to the output image. cmdline_file (Path): The path to the file containing the kernel command line.
Returns: A step to perform the operation.
@zbi_path.setter
— def zbi_path(self, path):
Sets the path to the zbi tool.
DEPS: auto_roller, git, recipe_engine/path, recipe_engine/properties, recipe_engine/raw_io
Example recipe for auto-rolling.
— def RunSteps(api, project, remote, commit_untracked_files, dry_run):
DEPS: bazel, tar, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/step
— def RunSteps(api, bazel_version):
DEPS: git, gitiles, go, goma, gsutil, recipe_engine/cipd, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/step, recipe_engine/time
Recipe for building Bloaty.
— def RunSteps(api, repository, branch, revision, platform):
— def cmake(api, cipd_dir, src_dir, platform, options=[], step_name=‘cmake’):
— def platform_sysroot(api, cipd_dir, platform):
— def upload_package(api, pkg_name, pkg_file, pkg_dir, repository, revision):
DEPS: git, gitiles, recipe_engine/cipd, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/step
Recipe for building some Breakpad tools.
— def RunSteps(api, url, ref):
DEPS: buildsetlookup, recipe_engine/path, recipe_engine/raw_io
— def RunSteps(api):
DEPS: catapult, recipe_engine/file, recipe_engine/json, recipe_engine/raw_io
— def RunSteps(api):
DEPS: checkout, recipe_engine/buildbucket, recipe_engine/json, recipe_engine/path, recipe_engine/properties
— def RunSteps(api, project, override, checkout_from_snapshot):
DEPS: auto_roller, git, recipe_engine/cipd, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/step
Recipe for rolling chromium prebuilts into Fuchsia.
— def GetVersionsFromCIPD(api, cipd_pkg):
— def RunSteps(api, dry_run):
DEPS: clang_tidy, fuchsia, git, goma, recipe_engine/buildbucket, recipe_engine/context, recipe_engine/json, recipe_engine/path, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/step, recipe_engine/tricium
Recipe for running CQ-blocking clang-tidy checks.
— def RunSteps(api, project, manifest, remote, checks):
DEPS: clang_tidy, fuchsia, recipe_engine/buildbucket, recipe_engine/json, recipe_engine/raw_io, recipe_engine/step
— def RunSteps(api):
DEPS: git, gitiles, goma, gsutil, recipe_engine/cipd, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/raw_io, recipe_engine/step, recipe_engine/tempfile
Recipe for building Clang toolchain.
— def RunSteps(api, repository, branch, revision, platform):
DEPS: cloudkms, recipe_engine/path, recipe_engine/raw_io
— def RunSteps(api):
DEPS: checkout, recipe_engine/buildbucket, recipe_engine/context, recipe_engine/path, recipe_engine/properties, recipe_engine/step
Recipe for building and testing Cobalt.
— def RunSteps(api, manifest, remote):
DEPS: git, lkgs, recipe_engine/context, recipe_engine/path, recipe_engine/properties, recipe_engine/time
Recipe for automatically cutting new release branches.
— def RunSteps(api, release_repository, release_names, release_targets, reference_builders):
DEPS: auto_roller, git, gitiles, jiri, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/properties, recipe_engine/python, recipe_engine/raw_io, recipe_engine/step, recipe_engine/tempfile
Recipe for automatically updating Flutter, flutter/engine, and Dart.
— def ExtractDartVersionFromDEPS(api, deps_path):
Extracts the dart_version from a flutter/engine's DEPS file.
Args: api (RecipeApi): Recipe API object. deps_path (Path): A path to the DEPS file for dart third party dependencies. manifest_path (Path): A path to the Jiri manifest to overwrite.
— def RollChanges(api, path, updated_deps):
Rolls manifest changes in a git repository.
Args: path (Path): Path to the git repository containing the changes to roll. updated_deps (dict[str]str): A map of dependencies that were updated to a log string, summarizing the update.
— def RunSteps(api, revision):
— def UpdateManifestProject(api, manifest, project_name, revision):
Updates the revision for a project in a manifest.
Args: api (RecipeApi): Recipe API object. manifest (Path): Path to the Jiri manifest to update. project_name (str): Name of the project in the Jiri manifest to update. revision (str): SHA-1 hash representing the updated revision for project_name in the manifest.
Returns: A formatted log string summarizing the updates as well as the project's remote property.
— def UpdatePkgManifest(api, dart_path, manifest_path):
Overwrites a dart third party package manifest.
Args: api (RecipeApi): Recipe API object. dart_path (Path): A path to the dart/sdk repository. manifest_path (Path): A path to the Jiri manifest to overwrite.
DEPS: auto_roller, checkout, git, jiri, recipe_engine/context, recipe_engine/json, recipe_engine/path, recipe_engine/properties, recipe_engine/python, recipe_engine/raw_io, recipe_engine/step
Recipe for automatically updating Dart 3p packages.
— def RunSteps(api):
DEPS: git, gitiles, goma, recipe_engine/cipd, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/step
Recipe for building Dart toolchain.
— def RunSteps(api, repository, branch, revision, host_cpu, host_os):
DEPS: auto_roller, fuchsia, jiri, recipe_engine/buildbucket, recipe_engine/cipd, recipe_engine/context, recipe_engine/json, recipe_engine/path, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/step
Recipe for generating docs.
— def RunSteps(api, project, manifest, remote, packages, run_gndoc):
— def gen_gndoc(api, packages, project_dir):
DEPS: fuchsia, gsutil, jiri, recipe_engine/buildbucket, recipe_engine/cipd, recipe_engine/context, recipe_engine/file, recipe_engine/path, recipe_engine/properties, recipe_engine/python, recipe_engine/step
Recipe for building FFmpeg and uploading it and required source files.
— def RunSteps(api, project):
DEPS: auto_roller, fuchsia, git, jiri, recipe_engine/buildbucket, recipe_engine/cipd, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/step
Recipe for generating docs for upload to Firebase.
— def RunSteps(api, dry_run, project, manifest, remote, target, build_type, packages, generators):
— def gen_dartdoc(api, out_dir, docs_dir, cipd_dir):
Generate dartdoc output.
Dartdoc runs on a single package, but has the capability to generate docs for all dependencies. Thus, to generate Dart documentation for Fuchsia, we first generate a ‘fake’ package that lists the libraries we want documented. We then run pub over that new package to fetch the dependencies, and finally dartdoc to generate documentation for it all.
Args: out_dir (Path) - The output directory for generated files. docs_dir (Path) - The output directory for documentation. cipd_dir (Path) - The cipd directory.
— def gen_rustdoc(api, out_dir, docs_dir, cipd_dir, build_dir):
Generate rust output.
The rustdoc script runs on GN targets. We find the Rust GN targets by finding all targets that generate a Cargo.toml file, and use those.
Args: out_dir (Path) - The output directory for generated files. docs_dir (Path) - The output directory for documentation. cipd_dir (Path) - The cipd directory.
DEPS: fuchsia, gsutil, hash, jiri, tar, testsharder, recipe_engine/buildbucket, recipe_engine/cipd, recipe_engine/context, recipe_engine/file, recipe_engine/path, recipe_engine/properties, recipe_engine/python, recipe_engine/step
Recipe for building Fuchsia and running tests.
— def RunSteps(api, project, manifest, remote, repo, checkout_snapshot, target, build_type, packages, variants, gn_args, ninja_targets, board, product, zircon_args, include_breakpad_symbols, include_symbol_archive, test_in_shards, run_tests, runtests_args, run_host_tests, test_pool, environment_label, device_type, networking_for_tests, pave, test_timeout_secs, requires_secrets, gcs_bucket):
DEPS: fuchsia, goma, jiri, swarming, testsharder, recipe_engine/buildbucket, recipe_engine/json, recipe_engine/path, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/step
Recipe for building Fuchsia and running tests.
— def RunSteps(api, project, manifest, remote, checkout_snapshot, target, build_type, packages, variants, gn_args, ninja_targets, run_tests, runtests_args, device_type, run_host_tests, networking_for_tests, requires_secrets, pave, board, product, zircon_args, test_in_shards, gcs_bucket, host_cmd):
DEPS: fuchsia, recipe_engine/buildbucket
Recipe for building Fuchsia and running connectivity tests.
TODO(IN-831): This will be replaced when host-target-interaction infra is in place.
This differs from the fuchsia recipe in the following ways:
— def RunSteps(api):
DEPS: catapult, fuchsia, minfs, swarming, recipe_engine/buildbucket, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/step, recipe_engine/time
Recipe for building Fuchsia and running performance tests.
This differs from the fuchsia recipe in the following ways:
— def RunSteps(api, project, manifest, remote, target, build_type, packages, variants, gn_args, ninja_targets, test_pool, catapult_url, device_type, pave, dashboard_masters_name, dashboard_bots_name, upload_to_dashboard, benchmarks_package, board, product, test_timeout_secs):
DEPS: auto_roller, gitiles, jiri, recipe_engine/context, recipe_engine/json, recipe_engine/path, recipe_engine/properties, recipe_engine/step
Recipe for rolling Fuchsia layers into upper layers.
— def RunSteps(api, project, manifest, remote, roll_type, import_in, import_from, revision, dry_run):
— def SsoToHttps(remote):
Transform a url of SSO scheme to its associated https version
DEPS: git, gitiles, goma, hash, jiri, recipe_engine/cipd, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/step, recipe_engine/tempfile, recipe_engine/url
Recipe for building GCC toolchain.
— def DoCheckout(api, url, checkout_dir, revision, last_revision, useless_file):
— def RunSteps(api, binutils_revision, gcc_revision):
DEPS: gerrit, recipe_engine/json, recipe_engine/properties
— def RunSteps(api):
DEPS: git, recipe_engine/context, recipe_engine/file, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/step, recipe_engine/time
— def RunSteps(api):
DEPS: gitiles, recipe_engine/properties
— def RunSteps(api):
DEPS: gn, goma, recipe_engine/path
— def RunSteps(api):
DEPS: go, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/step
— def RunSteps(api):
DEPS: git, gitiles, go, gsutil, recipe_engine/cipd, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/step
Recipe for building Go toolchain.
— def RunSteps(api, repository, branch, revision, platform):
DEPS: goma, recipe_engine/json, recipe_engine/platform, recipe_engine/properties, recipe_engine/step
— def RunSteps(api):
DEPS: gsutil, recipe_engine/path
— def RunSteps(api):
DEPS: hash, recipe_engine/path
— def RunSteps(api):
DEPS: checkout, git, go, jiri, recipe_engine/buildbucket, recipe_engine/cipd, recipe_engine/context, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/step, recipe_engine/url
Recipe for building and publishing infra tools.
This recipe builds one or more Go binaries in the specified project and publishes them all to CIPD. If one or more tests for any package in the project fail, or one or more packages fail to build, execution stops and no packages are uploaded.
This recipe uses golang/dep to manage dependencies, so the given project is expected to have a Gopkg.toml file specifying its dependency restrictions.
— def RunSteps(api, cipd_path, project, manifest, remote, packages):
— def UploadPackage(api, bin_name, bin_dir, cipd_path, revision, remote, platform):
Creates and uploads a CIPD package containing the tool at bin_dir/bin_name.
The tool is published to CIPD under the path ‘$cipd_path/$bin_name/$platform’
Args: bin_dir: The absolute path to the parent directory of bin_name. bin_name: The name of the tool binary cipd_path: The CIPD base path to use for package uploads
DEPS: isolated, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/step
— def RunSteps(api):
DEPS: checkout, git, go, gsutil, jiri, recipe_engine/buildbucket, recipe_engine/cipd, recipe_engine/context, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/step, recipe_engine/time
Recipe for building Jiri.
— def RunSteps(api, project, manifest, remote):
— def upload_package(api, name, platform, staging_dir, revision, remote):
DEPS: jiri, recipe_engine/buildbucket, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/step
— def RunSteps(api, tryjob, override):
DEPS: fuchsia, recipe_engine/buildbucket, recipe_engine/context, recipe_engine/path, recipe_engine/properties, recipe_engine/step
Recipe for checking license text in the source code.
Requires topaz source to be present in the manifest.
— def RunSteps(api, project, remote, manifest):
DEPS: lkgs, recipe_engine/path
— def RunSteps(api):
DEPS: git, gitiles, goma, gsutil, hash, recipe_engine/cipd, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/raw_io, recipe_engine/step, recipe_engine/tempfile
Recipe for building LLVM.
— def RunSteps(api, repository, branch, revision, platform):
DEPS: checkout, jiri, recipe_engine/buildbucket, recipe_engine/context, recipe_engine/file, recipe_engine/path, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/step
Recipe for testing LUCI configs.
— def RunSteps(api, config_project, manifest, remote):
DEPS: minfs, recipe_engine/path, recipe_engine/step
— def RunSteps(api):
DEPS: ninja, recipe_engine/path
— def RunSteps(api):
DEPS: git, recipe_engine/cipd, recipe_engine/context, recipe_engine/json, recipe_engine/path, recipe_engine/raw_io, recipe_engine/step
Recipe for building OVMF firmware and uploading them to CIPD.
The Open Virtual Machine Firmware (OVMF) project supports firmware for Virtual Machines (e.g., QEMU). The produced OVMF_CODE.fd and OVMF_VARS.fd are used to emulate EFI bootloading in QEMU in order to boot Zircon off of disk. Currently this is only supported for x64.
— def RunSteps(api):
DEPS: fuchsia, jiri, recipe_engine/buildbucket, recipe_engine/cipd, recipe_engine/context, recipe_engine/json, recipe_engine/properties, recipe_engine/step
Recipe for building and publishing CIPD prebuilts
— def GetBinPathComponents(build_dir, ninja_target):
Returns a binary's path based on its ninja target.
Args: build_dir: The Fuchsia build output directory ninja_target: A specific Ninja target that was built
Returns: bin_dir: The absolute path to the parent directory of bin_name. bin_name: The name of the tool binary
— def RunSteps(api, cipd_pkg_prefix, manifest, ninja_targets, packages, project, remote):
— def UploadPackage(api, bin_dir, bin_name, cipd_pkg_prefix, revision, remote):
Creates and uploads a CIPD package containing the tool at ‘$bin_dir/$bin_name’.
The tool is published to CIPD under the path ‘$cipd_pkg_prefix/$bin_name/$platform’
Args: api: The RecipeApi object bin_dir: The absolute path to the parent directory of bin_name. bin_name: The name of the tool binary cipd_pkg_prefix: The CIPD package prefix where the tool binary should be uploaded revision: The revision at which the tool binary was built remote: The git remote where code for the tool binary lives
DEPS: git, gitiles, goma, gsutil, recipe_engine/cipd, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/step, recipe_engine/tempfile
Recipe for building QEMU.
— def RunSteps(api, repository, branch, revision, platform):
— def build_gettext(api, cipd_dir, pkg_dir, platform, host):
— def build_glib(api, cipd_dir, pkg_dir, platform, host):
— def build_libffi(api, cipd_dir, pkg_dir, platform, host):
— def build_pixman(api, cipd_dir, pkg_dir, platform, host):
— def build_qemu(api, cipd_dir, pkg_dir, platform, host):
— def build_sdl(api, cipd_dir, pkg_dir, platform, env={}):
— def build_zlib(api, cipd_dir, pkg_dir, platform):
— def cmake(api, cipd_dir, src_dir, platform, options=[], step_name=‘cmake’):
— def configure(api, cipd_dir, src_dir, platform, host, flags=[], step_name=‘configure’):
— def platform_sysroot(api, cipd_dir, platform):
— def upload_package(api, pkg_name, pkg_dir, repository, revision):
DEPS: qemu, recipe_engine/path, recipe_engine/platform, recipe_engine/properties
— def RunSteps(api):
DEPS: checkout, recipe_engine/buildbucket, recipe_engine/context, recipe_engine/path, recipe_engine/properties, recipe_engine/python, recipe_engine/step
Recipe for testing Recipes.
— def RunSteps(api, project, manifest, remote):
DEPS: auto_roller, git, jiri, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/properties, recipe_engine/step, recipe_engine/time
Recipe for cherry-picking changes into a release.
— def RunSteps(api, branch, cherry_picks, pins, remote, gerrit_project, version):
DEPS: git, lkgs, recipe_engine/context, recipe_engine/path, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/time
Recipe for automatically creating jiri snapshots from manifest commit.
— def GetNextReleaseTag(api, remote):
— def RunSteps(api, branch, builders, remote):
DEPS: git, gitiles, jiri, recipe_engine/cipd, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/raw_io, recipe_engine/step, recipe_engine/url
Recipe for building Rust toolchain.
— def RunSteps(api, url, ref, revision):
DEPS: bazel, fuchsia, go, gsutil, hash, jiri, tar, recipe_engine/buildbucket, recipe_engine/cipd, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/step
Recipe for building Fuchsia SDKs.
— def RunSteps(api, project, manifest, remote, repo):
— def UploadArchive(api, gcs_archive_path, cipd_pkg_name, sdk, out_dir, remote, revision, upload_digest):
— def UploadPackage(api, sdk_name, staging_dir, remote, revision):
— def upload_raw_sdk(api, sdk_name, gcs_archive_path, cipd_pkg_name, archive_path, remote, revision, upload_digest):
DEPS: swarming, recipe_engine/json, recipe_engine/path, recipe_engine/properties
— def RunSteps(api, spawn_tasks):
DEPS: tar, recipe_engine/context, recipe_engine/file, recipe_engine/path, recipe_engine/platform, recipe_engine/step
— def RunSteps(api):
DEPS: testsharder, recipe_engine/path
— def RunSteps(api):
DEPS: jiri, recipe_engine/context, recipe_engine/path, recipe_engine/python
Recipe for checking licenses in the repo hosting third-party Rust crates.
— def RunSteps(api):
DEPS: checkout, git, go, jiri, recipe_engine/buildbucket, recipe_engine/cipd, recipe_engine/context, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/step, recipe_engine/url
Recipe for building and publishing tools.
— def RunSteps(api, project, manifest, remote, packages):
— def upload_package(api, name, platform, staging_dir, revision, remote):
DEPS: checkout, clang_tidy, git, jiri, recipe_engine/buildbucket, recipe_engine/context, recipe_engine/json, recipe_engine/path, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/step, recipe_engine/tricium
Recipe for running Tricium clang-format analyzer.
— def RunSteps(api, project, manifest, checks):
DEPS: checkout, git, recipe_engine/buildbucket, recipe_engine/cipd, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/properties, recipe_engine/python, recipe_engine/raw_io, recipe_engine/step, recipe_engine/tricium
Recipe for running Tricium FuchsiaFormat analyzer.
— def RunSteps(api, project, manifest, remote, formatters, name):
— def run_clang_format(api, cipd_dir, formatter):
Runs clang-format over the specified file and filters by changed line.
Since C/C++ formatting is so contentious, this special-cases the clang-format analysis by filtering it based on changed lines in the patch. Other formatters don't need this, since other languages standardize their formats.
Args: filename (str): Path to file. offset (int): Offset to convert.
DEPS: fuchsia, gitiles, gsutil, hash, jiri, recipe_engine/buildbucket, recipe_engine/context, recipe_engine/properties, recipe_engine/step
Recipe for building libwebkit.so.
— def RunSteps(api):
DEPS: zbi, recipe_engine/path, recipe_engine/raw_io, recipe_engine/step
— def RunSteps(api):
DEPS: fuchsia, goma, isolated, jiri, minfs, qemu, swarming, tar, zbi, recipe_engine/buildbucket, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/source_manifest, recipe_engine/step, recipe_engine/tempfile
Recipe for building Zircon.
— def Build(api, target, toolchain, make_args, src_dir, test_cmd, needs_blkdev, device_type):
Builds zircon and returns a path to the build output directory.
— def FinalizeTestsTasks(api, core_task, booted_task, booted_task_output_image, build_dir):
Waits on the tasks running core tests and booted tests, then analyzes the results.
Args: core_task (str): The swarming task ID of the task running core tests. booted_task (str): The swarming task ID of the task running booted tests. build_dir (Path): A path to the directory containing build artifacts.
— def RunSteps(api, project, manifest, remote, target, toolchain, make_args, use_kvm, run_tests, runtests_args, device_type, run_host_tests, gcs_bucket):
— def RunTestsInQEMU(api, target, build_dir, use_kvm):
Executes Zircon tests in QEMU on a different machine.
Args: api (recipe_engine.Api): The recipe engine API for this recipe. target (str): The zircon target architecture to execute tests on. build_dir (Path): Path to the build directory. use_kvm (bool): Whether or not to enable KVM with QEMU when testing.
Returns: A FuchsiaTestResults representing the completed test.
— def RunTestsOnDevice(api, target, build_dir, device_type):
Executes Zircon tests on a hardware device.
Args api (recipe_engine.Api): The recipe engine API for this recipe. target (str): The zircon target architecture to execute tests on. build_dir (Path): Path to the build directory. device_type (Enum(*DEVICES)): The type of device to run tests on. Returns: A FuchsiaTestResults representing the completed test.
— def RunTestsOnHost(api, build_dir):
Runs host tests.
Args: build_dir (Path): Path to the build directory. Returns: A FuchsiaTestResults representing the completed test.
— def TriggerTestsTask(api, name, cmd, arch, use_kvm, isolated_hash, output='', timeout_secs=(60 * 60)):
TriggerTestsTask triggers a task to execute a command on a remote machine.
The remote machine is guaranteed to have QEMU and botanist installed
Args: api: Recipe engine API object. name (str): Name of the task. cmd (seq[str]): The command to execute with each argument as a separate list entry. arch (str): The target architecture to execute tests for. use_kvm (bool): Whether or not a bot with KVM should be requested for the task. isolated_hash (str): A digest of the isolated containing the build artifacts. output (str): Optional relative path to an output file on the target machine which will be isolated and returned back to the machine executing this recipe. timeout_secs (int): The amount of seconds the task should run for before timing out.
Returns: The task ID of the triggered task.