DEPS: cipd, service_account, recipe_engine/context, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/raw_io, recipe_engine/step
AuthutilApi allows generating OAuth2 tokens from locally stored secrets.
This is a thin wrapper over the authutil go executable, which itself calls https://github.com/luci/luci-go/blob/master/client/authcli/authcli.go
— def ensure_authutil(self, version=None):
— def get_token(self, account, scopes=None, lifetime_sec=None):
DEPS: service_account, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/raw_io, recipe_engine/step
CIPDApi provides basic support for CIPD.
This assumes that cipd (or cipd.exe or cipd.bat on windows) has been installed somewhere in $PATH.
— def build(self, input_dir, output_package, package_name, install_mode=None):
Builds, but does not upload, a cipd package from a directory.
Args: input_dir (Path) - the directory to build the package from. output_package (Path) - the file to write the package to. package_name (str) - the name of the cipd package as it would appear when uploaded to the cipd package server. install_mode (None|‘copy’|‘symlink’) - the mechanism that the cipd client should use when installing this package. If None, defaults to the platform default (‘copy’ on windows, ‘symlink’ on everything else).
— def create_from_pkg(self, pkg_def, refs=None, tags=None):
Builds and uploads a package based on a PackageDefinition object.
This builds and uploads the package in one step.
Args: pkg_def (PackageDefinition) - The description of the package we want to create. refs (list(str)) - A list of ref names to set for the package instance. tags (dict(str, str)) - A map of tag name -> value to set for the package instance.
Returns the JSON ‘result’ section, e.g.: { “package”: “infra/tools/cipd/android-amd64”, “instance_id”: “433bfdf86c0bb82d1eee2d1a0473d3709c25d2c4” }
— def create_from_yaml(self, pkg_def, refs=None, tags=None):
Builds and uploads a package based on on-disk YAML package definition file.
This builds and uploads the package in one step.
Args: pkg_def (Path) - The path to the yaml file. refs (list(str)) - A list of ref names to set for the package instance. tags (dict(str, str)) - A map of tag name -> value to set for the package instance.
Returns the JSON ‘result’ section, e.g.: { “package”: “infra/tools/cipd/android-amd64”, “instance_id”: “433bfdf86c0bb82d1eee2d1a0473d3709c25d2c4” }
@property
— def default_bot_service_account_credentials(self):
— def describe(self, package_name, version, test_data_refs=None, test_data_tags=None):
— def ensure(self, root, packages):
Ensures that packages are installed in a given root dir.
packages must be a mapping from package name to its version, where
platform_suffix),If installing a package requires credentials, call set_service_account_credentials before calling this function.
@property
— def executable(self):
— def initialize(self):
— def platform_suffix(self, name=None, arch=None, bits=None):
Use to get full package name that is platform indepdent.
Example:
‘my/package/%s’ % api.cipd.platform_suffix() ‘my/package/linux-amd64’
Optional platform bits and architecture may be supplied to generate CIPD suffixes for other platforms. If any are omitted, the current platform parameters will be used.
— def register(self, package_name, package_path, refs=None, tags=None):
— def search(self, package_name, tag):
— def set_ref(self, package_name, version, refs):
— def set_service_account_credentials(self, path):
— def set_tag(self, package_name, version, tags):
DEPS: recipe_engine/context, recipe_engine/file, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/raw_io, recipe_engine/step
GitApi provides support for Git.
— def __call__(self, *args, **kwargs):
Return a git command step.
— def checkout(self, url, path=None, ref=None, remote=‘origin’, file=None, **kwargs):
Checkout a given ref and return the checked out revision.
— def commit(self, message, *files, **kwargs):
— def get_hash(self, commit=‘HEAD’, **kwargs):
Find and return the hash of the given commit.
— def get_timestamp(self, commit=‘HEAD’, test_data=None, **kwargs):
Find and return the timestamp of the given commit.
— def push(self, ref, remote=‘origin’, **kwargs):
— def rebase(self, branch=‘master’, remote=‘origin’, **kwargs):
Run rebase HEAD onto branch
DEPS: cipd, recipe_engine/context, recipe_engine/json, recipe_engine/path, recipe_engine/python, recipe_engine/raw_io, recipe_engine/step, recipe_engine/url
Module for polling a Git repository using the Gitiles web interface.
— def ensure_gitiles(self, version=None):
— def fetch(self, url, file_path, branch=‘master’, step_name=None, timeout=None, test_data=None):
Downloads raw file content from a Gitiles repository.
Args: url (str): base URL to the repository. file_path (str): relative path to the file from the repository root. branch (str): branch of the repository. step_name (str): custom name for this step (optional). timeout (int): number of seconds to wait before failing.
Returns: Raw file content.
— def log(self, url, treeish, limit=0, step_name=None, test_data={}):
Returns the most recent commits for treeish object.
Args: url (str): base URL of the remote repository. treeish (str): tree object identifier. limit (int): number of commits to limit the fetching to. step_name (str): custom name for this step (optional).
— def refs(self, url, refspath=‘refs/heads’, step_name=‘refs’, test_data=[]):
Resolves each ref in a repository to git revision
Args: url (str): URL of the remote repository. refspath (str): limits which refs to resolve.
DEPS: cipd, recipe_engine/context, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/python, recipe_engine/raw_io, recipe_engine/step
GoApi provides support for Go.
— def __call__(self, *args, **kwargs):
Return a Go command step.
— def ensure_go(self, version=None):
Ensures that go distribution is installed.
@property
— def go_executable(self):
@property
— def go_root(self):
— def inline(self, program, add_go_log=True, **kwargs):
Run an inline Go program as a step. Program is output to a temp file and run when this step executes.
DEPS: cipd, service_account, recipe_engine/context, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/raw_io, recipe_engine/step
GomaApi contains helper functions for using goma.
@contextmanager
— def build_with_goma(self, env={}):
Make context wrapping goma start/stop.
Raises: StepFailure or InfraFailure if it fails to build.
— def ensure_goma(self, canary=False):
@property
— def goma_ctl(self):
@property
— def goma_dir(self):
@contextmanager
— def goma_env(self, env):
@property
— def json_path(self):
@property
— def recommended_goma_jobs(self):
Return the recommended number of jobs for parallel build using Goma.
This function caches the _goma_jobs.
@property
— def service_account_json_path(self):
— def set_goma_dir(self, goma_dir):
This function is for local recipe test only.
— def start(self, env={}, **kwargs):
Start goma compiler_proxy.
A user MUST execute ensure_goma beforehand. It is user's responsibility to handle failure of starting compiler_proxy.
— def stop(self, env={}, **kwargs):
Stop goma compiler_proxy.
A user MUST execute start beforehand. It is user's responsibility to handle failure of stopping compiler_proxy.
Raises: StepFailure if it fails to stop goma.
DEPS: cipd, recipe_engine/context, recipe_engine/json, recipe_engine/path, recipe_engine/python, recipe_engine/step
GSUtilApi provides support for GSUtil.
— def __call__(self, *args, **kwargs):
Return a step to run arbitrary gsutil command.
— def copy(self, src_bucket, src, dst_bucket, dst, link_name=‘gsutil.copy’, unauthenticated_url=False, **kwargs):
— def ensure_gsutil(self, version=None):
@non_step
— def join(self, *parts):
Constructs a GS path from composite parts.
@non_step
— def normalize(self, url):
Normalize a GS URL using the gs:// URL prefix.
— def upload(self, bucket, src, dst, link_name=‘gsutil.upload’, unauthenticated_url=False, **kwargs):
DEPS: recipe_engine/path, recipe_engine/python, recipe_engine/raw_io
HashApi provides file hashing functionality.
— def md5(self, name, source, test_data=''):
— def sha1(self, name, source, test_data=''):
— def sha224(self, name, source, test_data=''):
— def sha256(self, name, source, test_data=''):
— def sha384(self, name, source, test_data=''):
— def sha512(self, name, source, test_data=''):
DEPS: cipd, recipe_engine/context, recipe_engine/json, recipe_engine/path, recipe_engine/raw_io, recipe_engine/step
APIs for interacting with isolates.
— def __call__(self, *args, **kwargs):
Return an isolate command step.
— def archive(self, isolate, isolated):
All the files in the .isolate file are put in the isolate server cache.
Args: isolate: .isolate file to load the dependency data from. isolated: .isolated file to generate or read.
— def ensure_isolate(self, version=None):
Ensures that isolate client is installed.
@property
— def isolate_client(self):
@isolate_server.setter
— def isolate_server(self, value):
Changes URL of Isolate server to use.
DEPS: cipd, recipe_engine/context, recipe_engine/json, recipe_engine/path, recipe_engine/raw_io, recipe_engine/source_manifest, recipe_engine/step
JiriApi provides support for Jiri managed checkouts.
— def __call__(self, *args, **kwargs):
Return a jiri command step.
— def checkout(self, manifest, remote, project=None, patch_ref=None, patch_gerrit_url=None, patch_project=None):
— def clean(self, all=False, **kwargs):
— def edit_manifest(self, manifest, projects=None, imports=None, test_data=None):
— def ensure_jiri(self, version=None):
— def import_manifest(self, manifest, remote, name=None, overwrite=False, **kwargs):
— def init(self, dir=None, **kwargs):
@property
— def jiri(self):
— def patch(self, ref, host=None, project=None, delete=False, force=False, rebase=False):
— def project(self, projects, test_data=None):
— def run_hooks(self, local_manifest=False):
— def snapshot(self, file=None, test_data=None, **kwargs):
— def source_manifest(self, file=None, test_data=None, **kwargs):
— def update(self, gc=False, rebase_tracked=False, local_manifest=False, run_hooks=True, snapshot=None, **kwargs):
DEPS: cipd, recipe_engine/context, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/raw_io, recipe_engine/step
QemuApi provides support for QEMU.
@contextmanager
— def background_run(self, *args, **kwargs):
— def ensure_qemu(self, version=None):
— def qemu_executable(self, arch):
— def run(self, step_name, *args, **kwargs):
ServiceAccountApi provides access to service account keys.
— def get_json_path(self, account):
DEPS: cipd, isolate, recipe_engine/context, recipe_engine/json, recipe_engine/path, recipe_engine/raw_io, recipe_engine/step
APIs for interacting with swarming.
— def __call__(self, *args, **kwargs):
Return a swarming command step.
— def collect(self, timeout, requests_json=None, tasks=[]):
Waits on a set of Swarming tasks.
Returns both the step result as well as a set of neatly parsed results.
Args: timeout: timeout to wait for result. requests_json: load details about the task(s) from the json file. tasks: list of task ids to wait on.
— def ensure_swarming(self, version=None):
Ensures that swarming client is installed.
@property
— def swarming_client(self):
@swarming_server.setter
— def swarming_server(self, value):
Changes URL of Swarming server to use.
— def trigger(self, name, raw_cmd, isolated=None, dump_json=None, dimensions=None, expiration=None, io_timeout=None, idempotent=False, cipd_packages=None):
Triggers a Swarming task.
Args: name: name of the task. raw_cmd: task command. isolated: hash of isolated test on isolate server. dump_json: dump details about the triggered task(s). dimensions: dimensions to filter slaves on. expiration: seconds before this task request expires. io_timeout: seconds to allow the task to be silent. idempotent: whether this task is considered idempotent. cipd_packages: list of 3-tuples corresponding to CIPD packages needed for the task: (‘path’, ‘package_name’, ‘version’), defined as follows: path: Path relative to the Swarming root dir in which to install the package. package_name: Name of the package to install, eg. “infra/tools/authutil/${platform}” version: Version of the package, either a package instance ID, ref, or tag key/value pair.
DEPS: cipd, recipe_engine/context, recipe_engine/path, recipe_engine/platform, recipe_engine/python, recipe_engine/step
Provides steps to tar and untar files.
— def create(self, archive, compression=None):
Returns TarPackage object that can be used to compress a set of files.
— def ensure_tar(self, version=None):
Ensures that bsdtar is installed.
— def extract(self, step_name, archive, dir=None, verbose=False):
Step to uncompress |tar_file| file.
DEPS: authutil, recipe_engine/json, recipe_engine/platform, recipe_engine/properties
— def RunSteps(api, scopes, lifetime_sec):
DEPS: cipd, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/step
— def RunSteps(api, use_pkg, pkg_files, pkg_dirs, ver_files, install_mode):
DEPS: cipd, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/step
— def RunSteps(api, arch_override, bits_override, expect_error):
DEPS: cipd, gsutil, hash, jiri, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/step, recipe_engine/tempfile
Recipe for building Clang toolchain.
— def RunSteps(api, category, patch_gerrit_url, patch_project, patch_ref, patch_storage, patch_repository_url, manifest, remote):
DEPS: cipd, jiri, recipe_engine/context, recipe_engine/path, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/step
Recipe for building and testing Cobalt.
— def RunSteps(api, patch_gerrit_url, patch_project, patch_ref, project, manifest, remote):
DEPS: cipd, goma, gsutil, jiri, qemu, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/source_manifest, recipe_engine/step
Builds the Fuchsia Dart test image and runs the Dart tests.
— def BuildFuchsia(api, build_type, target, gn_target, fuchsia_build_dir):
— def BuildZircon(api, zircon_project):
— def Checkout(api, manifest, remote):
— def RunSteps(api, manifest, remote, target, build_type, goma_dir):
— def RunTests(api, target, fuchsia_build_dir):
DEPS: cipd, goma, gsutil, hash, isolate, jiri, qemu, swarming, tar, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/source_manifest, recipe_engine/step
Recipe for building Fuchsia and running tests.
— def BuildFuchsia(api, build_type, target, gn_target, fuchsia_build_dir, modules, tests, use_autorun, use_isolate, gn_args):
— def BuildZircon(api, zircon_project):
— def Checkout(api, patch_project, patch_ref, patch_gerrit_url, project, manifest, remote):
— def IsolateArtifacts(api, target, zircon_build_dir, fuchsia_build_dir):
— def RunSteps(api, category, patch_gerrit_url, patch_project, patch_ref, patch_storage, patch_repository_url, project, manifest, remote, target, build_type, modules, tests, use_autorun, use_isolate, goma_dir, gn_args):
— def RunTestsInTask(api, target, isolated_hash):
— def RunTestsWithAutorun(api, target, fuchsia_build_dir):
— def RunTestsWithTCP(api, target, fuchsia_build_dir, tests):
DEPS: git, gitiles, jiri, recipe_engine/context, recipe_engine/path, recipe_engine/properties, recipe_engine/step
Recipe for rolling Fuchsia layers into upper layers.
— def RunSteps(api, category, project, manifest, remote, import_in, import_from, revision):
DEPS: git, recipe_engine/context, recipe_engine/file, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/step, recipe_engine/time
— def RunSteps(api):
DEPS: gitiles, recipe_engine/properties
— def RunSteps(api):
DEPS: go, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/step
— def RunSteps(api):
DEPS: cipd, go, gsutil, jiri, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/step, recipe_engine/tempfile
Recipe for building Go toolchain.
— def RunSteps(api, category, patch_gerrit_url, patch_project, patch_ref, patch_storage, patch_repository_url, manifest, remote):
DEPS: goma, recipe_engine/platform, recipe_engine/properties, recipe_engine/step
— def RunSteps(api):
DEPS: gsutil, recipe_engine/path
— def RunSteps(api):
DEPS: hash, recipe_engine/path
— def RunSteps(api):
DEPS: isolate, recipe_engine/json, recipe_engine/path
— def RunSteps(api):
DEPS: cipd, git, go, gsutil, jiri, recipe_engine/context, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/step, recipe_engine/time
Recipe for building Jiri.
— def RunSteps(api, category, patch_gerrit_url, patch_project, patch_ref, patch_storage, patch_repository_url, project, manifest, remote, target):
— def UploadPackage(api, revision, staging_dir):
DEPS: jiri, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/step
— def RunSteps(api):
DEPS: goma, jiri, recipe_engine/context, recipe_engine/path, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/step
Recipe for building and running pre-submit checks for the modules repo.
— def RunSteps(api, category, patch_gerrit_url, patch_project, patch_ref, patch_storage, patch_repository_url, project, manifest, remote, project_path):
DEPS: cipd, gsutil, jiri, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/step, recipe_engine/tempfile
Recipe for building QEMU.
— def RunSteps(api, category, patch_gerrit_url, patch_project, patch_ref, patch_storage, patch_repository_url, manifest, remote, platform):
DEPS: qemu, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/step
— def RunSteps(api, arch, kvm):
DEPS: jiri, recipe_engine/context, recipe_engine/path, recipe_engine/properties, recipe_engine/python, recipe_engine/step
Recipe for testing Recipes.
— def RunSteps(api, patch_gerrit_url, patch_project, patch_ref, project, manifest, remote):
DEPS: cipd, git, gitiles, gsutil, jiri, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/raw_io, recipe_engine/step, recipe_engine/url
Recipe for building Rust toolchain.
— def RunSteps(api, url, ref, revision):
DEPS: cipd, go, goma, gsutil, hash, jiri, recipe_engine/context, recipe_engine/file, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/step, recipe_engine/tempfile
Recipe for building Fuchsia SDKs.
— def BuildFuchsia(api, release_build, gn_target, fuchsia_build_dir, modules, use_goma, gn_args):
— def BuildZircon(api, project):
@contextmanager
— def GomaContext(api, use_goma):
— def MakeSdk(api, outdir, sdk):
— def PackageArchive(api, sdk):
— def RunSteps(api, category, patch_gerrit_url, patch_project, patch_ref, patch_storage, patch_repository_url, project, manifest, remote, use_goma, gn_args):
— def UploadArchive(api, sdk, digest):
— def UploadPackage(api, outdir, digest):
— def RunSteps(api):
DEPS: swarming, recipe_engine/json, recipe_engine/path
— def RunSteps(api):
DEPS: tar, recipe_engine/context, recipe_engine/file, recipe_engine/path, recipe_engine/platform, recipe_engine/step
— def RunSteps(api):
DEPS: jiri, recipe_engine/context, recipe_engine/path, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/step
Recipe for checking licenses in the repo hosting third-party Rust crates.
— def RunSteps(api, category, patch_gerrit_url, patch_project, patch_ref, patch_storage, patch_repository_url):
DEPS: goma, gsutil, jiri, recipe_engine/context, recipe_engine/path, recipe_engine/properties, recipe_engine/step
Recipe for building WebView.
— def RunSteps(api, category, patch_gerrit_url, patch_project, patch_ref, patch_storage, patch_repository_url, project, manifest, remote, target):
DEPS: cipd, goma, jiri, qemu, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/source_manifest, recipe_engine/step, recipe_engine/tempfile
Recipe for building Zircon.
— def RunSteps(api, category, patch_gerrit_url, patch_project, patch_ref, patch_storage, patch_repository_url, project, manifest, remote, target, toolchain, run_tests):
— def RunTests(api, name, build_dir, *args, **kwargs):
@contextlib.contextmanager
— def no_goma():