Shac (Scalable Hermetic Analysis and Checks) is a unified and ergonomic tool and framework for writing and running static analysis checks. The tool’s source can be found in the shac-documentation. Shac checks are written in Starlark.
Shac script implementations live in Fuchsia’s //scripts/shac directory.
rust.star, go.star, fidl.star). If it’s language specific but does not have a language.star file, then create one. If it’s generic, use title.star (where title is the name of the check function).The following example is a static analyzer on all files that creates a non-blocking, gerrit warning comment on changes where the string “http://” exists, pointing the user to use “https://” instead.
def http_links(ctx): for path, meta in ctx.scm.affected_files().items(): for num, line in meta.new_lines(): matches = ctx.re.allmatches(r"(http://)\w+", line) if not matches: continue for match in matches: ctx.emit.finding( message = "Avoid http:// links, prefer https://", # Change to "error" if the check should block presubmit. level = "warning", filepath = path, line = num, col = match.offset + 1, end_col = match.offset + 1 + len(match.groups[1]), replacements = ["https://"], )
Learn more about shac’s implementation of emit.findings.
Note that Shac does not automatically discover checks. In order for a check to run, a check function must be passed to shac.register_check() in //scripts/shac/main.star:
load("./http_links.star", "http_links") # NEW ... def register_all_checks(): ... shac.register_check(http_links) # NEW ...
When implementing a new check in a file that already contains other checks, you may be able to register the new check within that file. For example, //scripts/shac/fidl.star has a register_fidl_checks() function that gets called from //scripts/shac/main.star. Add new FIDL checks to fidl.star and register them in the register_fidl_checks() function in the same file.
Using a subprocess is useful if there’s an existing tool that does the check or if the logic of the check is complex (e.g. more than just a substring search). Starlark is intentionally feature-limited to encourage writing complicated business logic in a self-contained tool with its own unit tests.
The following is an example of a JSON formatter implemented in a separate Python script and run as a subprocess.
Rather than rewriting badly formatted files, the check computes the formatted contents and passes them to the replacements argument of the ctx.emit.finding() function. All formatting checks must be implemented this way, for the following reasons:
import json import sys def main(): # Accepts one positional argument referring to the file to format. path = sys.args[1] with open(path) as f: original = f.read() # Always use 2-space indents and a trailing blank line. formatted = json.dumps(json.loads(original), indent=2) + "\n" if formatted == original: sys.exit(0) else: print(json.dumps(doc, indent=2) + "\n") sys.exit(1) if __name__ == "__main__": main()
load("./common.star", "FORMATTER_MSG", "cipd_platform_name", "get_fuchsia_dir", "os_exec") def json_format(ctx): # Launch processes in parallel. procs = {} for f in ctx.scm.affected_files(): if not f.endswith(".json"): continue # Call fuchsia-specific `os_exec` function instead of # `ctx.os.exec()` to ensure proper executable resolution. # `os_exec` starts the subprocess but does not block. procs[f] = os_exec(ctx, [ "%s/prebuilt/third_party/python3/%s/bin/python3" % ( get_fuchsia_dir(ctx), cipd_platform_name(ctx), ), "scripts/shac/json_format.py", f, ]) for f, proc in procs.items(): # wait() blocks until the process completes. res = proc.wait() if proc.retcode != 0: ctx.emit.finding( level = "error", filepath = f, # FORMATTER_MSG is the standard message for formatters # in fuchsia.git. message = FORMATTER_MSG, # json_format.py prints the formatted file contents to stdout. # Passing it to `replacements` is necessary for shac to know # how to apply the fix. replacements = [res.stdout], ) # TODO: call this somewhere shac.register_check(shac.check( json_format, # Mark the check as a formatter. Only checks with `formatter = True` # get run by `fx format-code`. formatter = True, ))
Some formatters have built-in support for validating the formatting of many files at a time, which is often parallelized internally and therefore much faster than launching a separate subprocess to check every file. In this case, you can run the formatter once on all files in “check” mode to get a list of badly formatted files, and then iterate over only the badly formatted files to get the formatted result (as opposed to iterating over all files).
Example: for rustfmt first run rustfmt --check --files-with-diff <all rust files> to get a list of badly formatted files, then run rustfmt separately on each file to get the formatted result.
If the formatter does not have a dry-run mode to print the formatted result to stdout: The formatter subprocesses will not be able to write to the checkout. However, some formatters unconditionally write files. In this case, you'll need to copy each file into a tempdir, to which the subprocess can write, format the temp file, and report its contents, as an example see buildifier.
By default, os_exec raises an un-recoverable error if the subprocess produces a nonzero return code. If non-zero return codes are expected, you can use the ok_retcodes parameter, e.g. ok_retcodes = [0, 1] may be appropriate if the formatter produces a return code of 1 when the file is unformatted.
To run static analysis checks locally across modified files in your change, use fx lint:
fx lint runs all in-tree linters and analyzers on modified files.fx lint --fix automatically applies suggested replacements emitted by checks.fx lint --only <check_name> runs only a specific check.fx lint --all runs static analysis across all tracked files in the repository.During local check development it’s recommended to test your check by running fx lint or invoking shac directly via fx host-tool shac check <file>. Let’s create a scenario in which we can test the http_links check described above:
echo "http://example.com" > temp.txtfx host-tool shac check --only http_links temp.txt--only causes shac to only run the http_links check, excluding other checks because in this instance we only care about testing http_links and don't care about results from other checksfx host-tool shac fix --only http_links temp.txt should change the http:// to https://fx host-tool shac check --only http_links temp.txt Should now passfx host-tool shac check --only http_links --all//shac.textproto), not just changed filesfx host-tool shac check --all)It is recommended that you document your check if it is opt-in (not run in pre-submit) or there's a non-obvious opt-out mechanism. All documentation should be added to //docs/development/source_code/presubmit_checks.md
Checks should have unit tests, which are written in Starlark and run with shac test. A test runs a check against a small, temporary checkout whose files are specified by the test, so tests are fast and hermetic and don't depend on the contents of the real checkout.
Tests for the checks in //scripts/shac live in //scripts/shac/tests, in a <name>_test.star file named after the file containing the check (e.g. python_test.star tests checks in python.star). Every top-level function whose name starts with test_ is a test case. Test files can load() check functions directly, including private ones.
Here is an example of tests for the http_links check from the simple example:
load("//scripts/shac/http_links.star", "http_links") def test_http_links(): res = testing.run(http_links, files = { "foo.md": "See http://example.com.\n", }) asserts.eq(res.findings, ( testing.finding( level = "warning", message = "Avoid http:// links, prefer https://", filepath = "foo.md", line = 1, col = 5, end_col = 12, replacements = ["https://"], ), )) # `res.files` contains the file contents after applying all fixes. asserts.eq(res.files["foo.md"], "See https://example.com.\n") def test_http_links_ok(): res = testing.run(http_links, files = { "foo.md": "See https://example.com.\n", }) asserts.eq(res.findings, ())
testing.run() treats every file passed in files as affected by the change under test, unless affected_files is set. Checks that run subprocesses can either execute real prebuilt tools or mock them with testing.exec_mock(). See the shac-documentation for the full testing and asserts APIs.
Fuchsia's own checks also run on test files, so test data that would trigger a check on the test file itself (e.g. a confusing character or a bug URL) must be split up or escaped, e.g. "http" + "://" or "\u200b".
To add a new test file, add a host_shac_test() target for it in //scripts/shac/tests/BUILD.bazel, named after the file, and add it to the host_tests test suite in the same file. If the tests execute any real tools, also add those tools to the target's data. Each test runs in presubmit whenever the checks it depends on or the test file itself change.
To run tests locally, use any of the following:
fx test //scripts/shac/tests:python_test runs the tests in a single file.fx bazel test --config=host //scripts/shac/tests:host_tests runs all tests.fx host-tool shac test scripts/shac/tests/python_test.star, from the root of the checkout, runs the tests in a single file without building anything. Use --only <test_name> to run only specific test cases.