Package documentation for infra

authutil
cipd
git
go
goma
gsutil
hash
jiri
qemu
service_account
tar

authutil:examples/full
cipd:examples/full
clang_toolchain — Recipe for building Clang toolchain.
cobalt — Recipe for building and testing Cobalt.
fuchsia — Recipe for building Fuchsia and running tests.
git:examples/full
go:examples/full
go_toolchain — Recipe for building Go toolchain.
goma:examples/full
gsutil:examples/full
hash:examples/full
jiri — Recipe for building Jiri.
jiri:examples/full
magenta — Recipe for building Magenta.
modules — Recipe for building and running pre-submit checks for the modules repo.
qemu — Recipe for building QEMU.
qemu:examples/full
recipes — Recipe for testing Recipes.
rust_toolchain — Recipe for building Rust toolchain.
sdk — Recipe for building Fuchsia SDKs.
service_account:examples/full
tar:examples/full
third_party_rust_crates — Recipe for checking licenses in the repo hosting third-party Rust crates.
web_view — Recipe for building WebView.

Recipe Modules

recipe_modules / authutil

DEPS: cipd, service_account, recipe_engine/context, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/raw_io, recipe_engine/step

class AuthutilApi(RecipeApi):

AuthutilApi allows generating OAuth2 tokens from locally stored secrets.

This is a thin wrapper over the authutil go executable, which itself calls https://github.com/luci/luci-go/blob/master/client/authcli/authcli.go

— def ensure_authutil(self, version=None):

— def get_token(self, account, scopes=None, lifetime_sec=None):

recipe_modules / cipd

DEPS: service_account, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/raw_io, recipe_engine/step

class CIPDApi(RecipeApi):

CIPDApi provides support for CIPD.

— def build(self, input_dir, output_package, package_name, install_mode=None):

— def create(self, pkg_def, refs=None, tags=None):

Creates a package based on YAML package definition file.

This builds and uploads the package in one step.

@property
— def default_bot_service_account_credentials(self):

— def describe(self, package_name, version, test_data_refs=None, test_data_tags=None):

— def ensure(self, root, packages):

Ensures that packages are installed in a given root dir.

packages must be a mapping from package name to its version, where

name must be for right platform (see also platform_suffix),
version could be either instance_id, or ref, or unique tag.

If installing a package requires credentials, call set_service_account_credentials before calling this function.

— def platform_suffix(self):

Use to get full package name that is platform indepdent.

Example:

‘my/package/%s’ % api.cipd.platform_suffix() ‘my/package/linux-amd64’

— def register(self, package_name, package_path, refs=None, tags=None):

— def search(self, package_name, tag):

— def set_ref(self, package_name, version, refs):

— def set_service_account_credentials(self, path):

— def set_tag(self, package_name, version, tags):

recipe_modules / git

DEPS: recipe_engine/context, recipe_engine/file, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/raw_io, recipe_engine/step

class GitApi(RecipeApi):

GitApi provides support for Git.

— def __call__(self, *args, **kwargs):

Return a git command step.

— def checkout(self, url, path=None, ref=None, remote=None, file=None, **kwargs):

Checkout a given ref and return the checked out revision.

— def get_hash(self, commit=‘HEAD’, **kwargs):

Find and return the hash of the given commit.

— def get_timestamp(self, commit=‘HEAD’, test_data=None, **kwargs):

Find and return the timestamp of the given commit.

— def rebase(self, branch=‘master’, remote=‘origin’, **kwargs):

Run rebase HEAD onto branch

recipe_modules / go

DEPS: cipd, recipe_engine/context, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/python, recipe_engine/raw_io, recipe_engine/step

class GoApi(RecipeApi):

GoApi provides support for Go.

— def __call__(self, *args, **kwargs):

Return a Go command step.

— def ensure_go(self, version=None):

Ensures that go distribution is installed.

@property
— def go_executable(self):

@property
— def go_root(self):

— def inline(self, program, add_go_log=True, **kwargs):

Run an inline Go program as a step. Program is output to a temp file and run when this step executes.

recipe_modules / goma

DEPS: cipd, service_account, recipe_engine/context, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/raw_io, recipe_engine/step

class GomaApi(RecipeApi):

GomaApi contains helper functions for using goma.

@contextmanager
— def build_with_goma(self, env=None):

Make context wrapping goma start/stop.

Raises: StepFailure or InfraFailure if it fails to build.

— def ensure_goma(self, canary=False):

@property
— def goma_ctl(self):

@property
— def goma_dir(self):

@contextmanager
— def goma_env(self):

@property
— def json_path(self):

@property
— def recommended_goma_jobs(self):

Return the recommended number of jobs for parallel build using Goma.

This function caches the _goma_jobs.

@property
— def service_account_json_path(self):

— def start(self, env=None, **kwargs):

Start goma compiler_proxy.

A user MUST execute ensure_goma beforehand. It is user's responsibility to handle failure of starting compiler_proxy.

— def stop(self, **kwargs):

Stop goma compiler_proxy.

A user MUST execute start beforehand. It is user's responsibility to handle failure of stopping compiler_proxy.

Raises: StepFailure if it fails to stop goma.

recipe_modules / gsutil

DEPS: cipd, recipe_engine/context, recipe_engine/json, recipe_engine/path, recipe_engine/python, recipe_engine/step

class GSUtilApi(RecipeApi):

GSUtilApi provides support for GSUtil.

— def __call__(self, *args, **kwargs):

Return a step to run arbitrary gsutil command.

— def copy(self, src_bucket, src, dst_bucket, dst, link_name=‘gsutil.copy’, unauthenticated_url=False, **kwargs):

— def ensure_gsutil(self, version=None):

@non_step
— def join(self, *parts):

Constructs a GS path from composite parts.

@non_step
— def normalize(self, url):

Normalize a GS URL using the gs:// URL prefix.

— def upload(self, bucket, src, dst, link_name=‘gsutil.upload’, unauthenticated_url=False, **kwargs):

recipe_modules / hash

DEPS: recipe_engine/path, recipe_engine/python, recipe_engine/raw_io

class HashApi(RecipeApi):

HashApi provides file hashing functionality.

— def md5(self, name, source, test_data=''):

— def sha1(self, name, source, test_data=''):

— def sha224(self, name, source, test_data=''):

— def sha256(self, name, source, test_data=''):

— def sha384(self, name, source, test_data=''):

— def sha512(self, name, source, test_data=''):

recipe_modules / jiri

DEPS: cipd, recipe_engine/context, recipe_engine/json, recipe_engine/path, recipe_engine/raw_io, recipe_engine/step

class JiriApi(RecipeApi):

JiriApi provides support for Jiri managed checkouts.

— def __call__(self, *args, **kwargs):

Return a jiri command step.

— def clean(self, all=False, **kwargs):

— def ensure_jiri(self, version=None):

— def import_manifest(self, manifest, remote, overwrite=False, **kwargs):

— def init(self, dir=None, **kwargs):

@property
— def jiri(self):

— def patch(self, ref, host=None, delete=False, force=False, rebase=False, **kwargs):

— def project(self, *projects, **kwargs):

— def snapshot(self, file, step_test_data=None, **kwargs):

— def update(self, gc=False, snapshot=None, local_manifest=False, **kwargs):

recipe_modules / qemu

DEPS: cipd, recipe_engine/context, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/python, recipe_engine/raw_io, recipe_engine/step

class QemuApi(RecipeApi):

QemuApi provides support for QEMU.

@contextmanager
— def background_run(self, *args, **kwargs):

— def ensure_qemu(self, version=None):

— def qemu_executable(self, arch):

— def run(self, step_name, *args, **kwargs):

recipe_modules / service_account

DEPS: recipe_engine/path

class ServiceAccountApi(RecipeApi):

ServiceAccountApi provides access to service account keys.

— def get_json_path(self, account):

recipe_modules / tar

DEPS: cipd, recipe_engine/context, recipe_engine/path, recipe_engine/platform, recipe_engine/python, recipe_engine/step

class TarApi(RecipeApi):

Provides steps to tar and untar files.

— def create(self, archive, compression=None):

Returns TarPackage object that can be used to compress a set of files.

— def ensure_tar(self, version=None):

Ensures that bsdtar is installed.

— def extract(self, step_name, archive, dir=None, verbose=False):

Step to uncompress |tar_file| file.

Recipes

recipes / authutil:examples/full

DEPS: authutil, recipe_engine/json, recipe_engine/platform, recipe_engine/properties

— def RunSteps(api, scopes, lifetime_sec):

recipes / cipd:examples/full

DEPS: cipd, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/step

— def RunSteps(api):

recipes / clang_toolchain

DEPS: cipd, gsutil, jiri, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/step, recipe_engine/tempfile

Recipe for building Clang toolchain.

— def RunSteps(api, category, patch_gerrit_url, patch_project, patch_ref, patch_storage, patch_repository_url, manifest, remote):

recipes / cobalt

DEPS: jiri, recipe_engine/context, recipe_engine/path, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/step

Recipe for building and testing Cobalt.

— def RunSteps(api, patch_gerrit_url, patch_ref, manifest, remote):

recipes / fuchsia

DEPS: cipd, goma, gsutil, hash, jiri, qemu, tar, recipe_engine/context, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/step

Recipe for building Fuchsia and running tests.

— def BuildFuchsia(api, release_build, target, gn_target, fuchsia_build_dir, modules, boot_module, tests, use_goma, gn_args):

— def BuildMagenta(api, target):

— def Checkout(api, patch_project, patch_ref, patch_gerrit_url, manifest, remote):

@contextmanager
— def GomaContext(api, use_goma):

— def RunSteps(api, category, patch_gerrit_url, patch_project, patch_ref, patch_storage, patch_repository_url, manifest, remote, target, build_type, modules, boot_module, tests, use_goma, gn_args):

— def RunTests(api, target, fuchsia_build_dir, tests):

— def UploadArchive(api, target, magenta_build_dir, fuchsia_build_dir):

recipes / git:examples/full

DEPS: git, recipe_engine/context, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/step

— def RunSteps(api):

recipes / go:examples/full

DEPS: go, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/step

— def RunSteps(api):

recipes / go_toolchain

DEPS: cipd, go, gsutil, jiri, recipe_engine/context, recipe_engine/file, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/step, recipe_engine/tempfile

Recipe for building Go toolchain.

— def RunSteps(api, category, patch_gerrit_url, patch_project, patch_ref, patch_storage, patch_repository_url, manifest, remote):

recipes / goma:examples/full

DEPS: goma, recipe_engine/platform, recipe_engine/properties, recipe_engine/step

— def RunSteps(api):

recipes / gsutil:examples/full

DEPS: gsutil, recipe_engine/path

— def RunSteps(api):

recipes / hash:examples/full

DEPS: hash, recipe_engine/path

— def RunSteps(api):

recipes / jiri

DEPS: cipd, git, go, gsutil, jiri, recipe_engine/context, recipe_engine/json, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/step, recipe_engine/time

Recipe for building Jiri.

— def RunSteps(api, category, patch_gerrit_url, patch_project, patch_ref, patch_storage, patch_repository_url, manifest, remote, target):

— def UploadPackage(api, revision, staging_dir):

recipes / jiri:examples/full

DEPS: jiri, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/step

— def RunSteps(api):

recipes / magenta

DEPS: cipd, jiri, qemu, recipe_engine/context, recipe_engine/file, recipe_engine/path, recipe_engine/platform, recipe_engine/properties, recipe_engine/raw_io, recipe_engine/step, recipe_engine/tempfile

Recipe for building Magenta.

— def RunSteps(api, category, patch_gerrit_url, patch_project, patch_ref, patch_storage, patch_repository_url, manifest, remote, target, toolchain, run_tests):

— def RunTests(api, name, *args, **kwargs):