commit a3e772d91395b331e8ea679d86dc676aa2101d2c
author Nodir Turakulov <nodir+github@google.com> Wed Jun 09 19:17:02 2021 +0000
committer Nodir Turakulov <nodir+github@google.com> Wed Jun 23 17:26:17 2021 +0000
tree 0770ec9acd936cb336099f544035a19fe2c1cb00
parent b2689fabc306d2cd20356f30b7f99eec445d9212

[cas] Add PathSpec.Allowlist

In the Isolate use case, we need to upload a directory partially.
Currently cas package does not let doing it efficiently.

It is possible to upload individual files/directories, but then the
caller has to construct the partial merkle tree themselves. For example,
to upload "/foo" with only "/foo/bar" and "/foo/baz", but not
"/foo/qux", the client has to upload bar and baz, and then calculate the
digest of /foo manually.

It is possible to exclude some paths using UploadOptions.Prelude,
but this filtering is at the UploadOptions level, while we need it at
the PathSpec level. Placing a callback in the PathSpec level makes
caching of intermediate directories difficult. In Go, it is not possible
to tell if two function pointers represent the same function with same
captured input.

Introduce PathSpec.Allowlist which is a list of paths, relative to
PathSpec.Path, that must be uploaded. The client uploads them the usual
way, and then constructs a partial Merkle tree to upload the root, as
well as the intermediate directories between the root and the
allowlisted paths.

The allowlist does not participate in the cache key. Doing so is
possible, but would require hashing the allowlist for each PathSpec and
might be an overkill *for now*. Also it complicates the documentation.
Instead, document that if a given PathSpec pointer was passed to
Client.Upload() earlier, then the same pointer is guaranteed to work in
UploadResult.Digest(). This is in addition to structural cache key which
is used to retrieve digests of files that were uploaded implicitly.