Shac uses the starlark language. Starlark is a python derivative. https://bazel.build/rules/language is a great resource if the language is new to you, just ignore the bazel references. The starlark language formal specification is documented at https://github.com/google/starlark-go/blob/HEAD/doc/spec.md.
While all starlark-go's built-in constants and functions are available, a few are explicitly documented here to highlight them.
These experimental features are enabled:
ctx is the object passed to shac.register_check(...) callback.
Fields:
ctx.emit is the object that exposes the API to emit results for checks.
Fields:
Emits a finding from the current check.
A check level finding:
def cb(ctx): ctx.emit.finding(level="warning", message="Do not change anything") shac.register_check(cb)
A finding associated with a specific file:
def cb(ctx): for path, _ in ctx.scm.affected_files().items(): ctx.emit.finding( level="notice", message="great code", filepath=path, ) shac.register_check(cb)
A finding associated with a specific line within a file:
def cb(ctx): for path, meta in ctx.scm.affected_files().items(): for num, line in meta.new_lines(): ctx.emit.finding( level="error", message="This line is superfluous", filepath=path, line=num, # Suggesting an empty string as a replacement results in the line # being deleted when applying the replacement. replacements=[""], ) shac.register_check(cb)
A finding associated with a line and column within a file:
def cb(ctx): for path, meta in ctx.scm.affected_files().items(): for num, line in meta.new_lines(): idx = str.find("bad_word") if idx < 0: continue ctx.emit.finding( level="error", message="Do not use bad_word", filepath=path, line=num, start_col=idx+1, end_col=idx+1+len("bad_word"), replacements=["best_word", "good_word"], ) shac.register_check(cb)
Emits an artifact from the current check.
def cb(ctx): ctx.emit.artifact("result.txt", "fake data") shac.register_check(cb)
ctx.io is the object that exposes the API to interact with the file system.
Fields:
Returns the content of a file.
def cb(ctx): # Read at most 4Kib of "path/to/file.txt". content = str(ctx.io_read_file("path/to/file.txt", size=4096)) # Usually run a regexp via ctx.re.match(), or other simple text # processing. print(content) shac.register_check(cb)
Content of the file as bytes.
Returns a new temporary directory.
ctx.os is the object that exposes the API to interact with the operating system.
Fields:
ctx.os.name contains the OS as described by GOOS. Frequent values are “windows”, “linux” and “darwin”. The full exact list can be retrieved with the command “go tool dist list | cut -f 1 -d / | uniq”
Runs a command as a subprocess.
Subprocesses are denied network access by default on Linux. Use allow_network = True
to grant the subprocess network access.
def cb(ctx): res = ctx.os.exec(["echo", "hello world"], cwd=".").wait() print(res.stdout) # "hello world" shac.register_check(cb)
Use raise_on_failure = False
to prevent a non-zero retcode from automatically failing the check:
def cb(ctx): res = ctx.os.exec(["cat", "does-not-exist.txt"], raise_on_failure = False).wait() print(res.retcode) # 1 print(res.stderr) # cat: does-not-exist.txt: No such file or directory shac.register_check(cb)
Use env
to pass environment variables:
def cb(ctx): ctx.os.exec(["foo"], env = {"FOO_CONFIG": "foo.config"}).wait() shac.register_check(cb)
A subprocess object with a wait() method. wait() returns a struct(retcode=..., stdout=“...”, stderr=“...”)
ctx.re is the object that exposes the API to run regular expressions on starlark strings.
Fields:
Returns all the matches of the regexp pattern onto content.
def cb(ctx): content = str(ctx.io_read_file("path/to/file.txt")) for match in ctx.re.allmatches("TODO\(([^)]+)\).*", content): print(match) shac.register_check(cb)
list(struct(offset=bytes_offset, groups=list(matches)))
Returns the first match of the regexp pattern onto content.
def cb(ctx): content = str(ctx.io_read_file("path/to/file.txt")) # Only print the first match, if any. match = ctx.re.match("TODO\(([^)]+)\).*", "content/true") print(match) shac.register_check(cb)
struct(offset=bytes_offset, groups=list(matches))
ctx.scm is the object exposes the API to query the source control management (e.g. git).
Fields:
ctx.scm.root is the absolute path to the project root.
Returns affected files as determined by the SCM.
If shac detected that the tree is managed by a source control management system, e.g. git, it will detect the upstream branch and return only the files currently modified.
If the current directory is not controlled by a SCM, the result is equivalent to ctx.scm.all_files().
If shac is run with the --all options, all files are considered “added” to do a full run on all files.
def new_todos(cb): # Prints only the TODO that were added compared to upstream. for path, meta in ctx.scm.affected_files().items(): for num, line in meta.new_lines(): m = ctx.re.match("TODO\(([^)]+)\).*", line) print(path + "(" + str(num) + "): " + m.groups[0]) shac.register_check(new_todos)
A map of {path: struct()} where the struct has a string field action and a function new_lines().
Returns all files found in the current workspace.
All files are considered “added” or “deleted”.
def all_todos(cb): for path, meta in ctx.scm.all_files().items(): for num, line in meta.new_lines(): m = ctx.re.match("TODO\(([^)]+)\).*", line) print(path + "(" + str(num) + "): " + m.groups[0]) shac.register_check(all_todos)
A map of {path: struct()} where the struct has a string field action and a function new_lines().
Starlark builtin that returns all the attributes of an object.
Primarily used to explore and debug a starlark file.
See the official documentation at https://github.com/google/starlark-go/blob/HEAD/doc/spec.md#dir.
def print_attributes(name, obj): for attrname in dir(obj): attrval = getattr(obj, attrname) attrtype = type(attrval) fullname = name + "." + attrname if attrtype in ("builtin_function_or_method", "function"): print(fullname + "()") elif attrtype == "struct": print_attributes(fullname, attrval) else: print(fullname + "=" + repr(attrval)) def cb(ctx): print_attributes("ctx", ctx) print_attributes("str", "") print_attributes("dict", {}) print_attributes("set", set()) print_attributes("struct", struct(foo = "bar", p = print_attributes)) shac.register_check(cb)
List of x object properties as strings. You can use getattr() to retrieve each attributes in a loop.
Starlark builtin that fails immediately the execution.
This function will abort execution. When called in the first phase, outside a check callback, it will prevent execution of checks.
When called within a check, it stops the check execution and annotates it with an abnormal failure. It also prevents checks that were not yet started from running.
See the official documentation at https://github.com/google/starlark-go/blob/HEAD/doc/spec.md#fail.
The code after the fail() call will not be executed:
fail("implement me")
The check will be annotated with the abnormal failure:
def cb1(ctx): fail("implement me") def cb2(ctx): # This check may or may not run, depending on concurrency. pass shac.register_check(cb1) shac.register_check(cb2)
json is a global module that exposes json functions.
The documentation here is listed as a struct instead of a module. The two are functionally equivalent.
The implementation matches the official bazel's documentation at https://bazel.build/rules/lib/json except that encode_indent is not implemented.
Fields:
Decodes a JSON encoded string into the Starlark value that the string denotes.
Supported types include null, bool, int, float, str, dict and list.
See the full documentation at https://bazel.build/rules/lib/json#decode.
data = json.decode('{"foo":"bar}') print(data["foo"]) def cb(ctx): # Load a configuration from a json file in the tree, containing a # dict with a "version" key. decoded = ctx.io.read_file("config.json") print(decoded["version"]) shac.register_check(cb)
Encodes the starlark value into a JSON encoded string.
Supported types include null, bool, int, float, str, dict, list and struct.
See the full documentation at https://bazel.build/rules/lib/json#encode.
config = struct( foo = "bar", ) print(json.encode(config))
Returns the indented form of a valid JSON-encoded string.
See the full documentation at https://bazel.build/rules/lib/json#indent.
config = struct( foo = "bar", ) d = json.encode(config) print(json.indent(d))
Starlark builtin that loads an additional shac starlark package and make symbols (var, struct, functions) from this file accessible.
See the official documentation at https://github.com/google/starlark-go/blob/HEAD/doc/spec.md#name-binding-and-variables and at https://github.com/google/starlark-go/blob/HEAD/doc/spec.md#load-statements.
After a starlark module is loaded, its values are frozen as described at https://github.com/google/starlark-go/blob/HEAD/doc/spec.md#freezing-a-value.
There are 5 forms for the module to load:
load("../common/go.star", "gosec") register_check(gosec)
load("//common/go.star", "gosec") register_check(gosec)
# Implicitly loads api.star load("@static-checks", "go") register_check(go.gosec)
load("@static-checks//go.star", "go") register_check(go.gosec)
# Implicitly loads api.star load("@go.fuchsia.dev/shac-project/static-checks", "gosec") # or load("@go.fuchsia.dev/shac-project/static-checks//go.star", "gosec") register_check(gosec)
Starlark builtin that prints a debug log.
This function should only be used while debugging the starlark code. See the official documentation at https://github.com/google/starlark-go/blob/HEAD/doc/spec.md#print.
print("shac", "is", "great")
shac is the global available at runtime when loading your starlark code.
Fields:
The git hash of shac.git where shac was built.
The semver version number of shac.
Constructs a shac check object.
def cb(ctx): fail("implement me") fail_often = shac.check(cb, name="fail_often") shac.register_check(fail_often)
shac fmt
.Registers a shac check.
It must be called at least once for the starlark file to be a valid check file. Each callback will be run in parallel. Each check must have a different name.
def cb(ctx): fail("implement me") fail_often = shac.check(cb, name="fail_often") shac.register_check(fail_often)
register_check also accepts a bare function for convenience when registering simple checks. The callback function name will be used as the check name.
def fail_often(ctx): fail("implement me") shac.register_check(cb, fail_often)
shac.check()
object or Starlark function that is called back to implement the check.Creates and return a structure instance.
Create an “object” that has immutable properties. It is similar to a dictionary is usage. It is intentionally not as powerful as a python class instance.
def _do(): print("it works") obj = struct( value = "a value", do = _do, ) print(obj.value) obj.do()