Development Report for July 17, 2017

BuildKit

Following features were added last week:

Git source

Source code from git repositories can now be accessed directly, similarly for images can be accessed, without the need to execute git clone. This has many performance and caching advantages. It accesses the remote repository using shallow fetches to only pull the required data and a uses a shared bare repository for intermediate cache between build invocations. The instruction cache for the git source is based on a commit hash and not string arguments. This means that you can always be sure that you are building the correct source and that you never build the same source twice.

Containerd exporter

Exporters are used for getting build artifacts out of buildkit. The first exporter that was implemented allows exposing the image to containerd so it can be run and pushed with ctr tool. buildctl has --exporter flag for specifying the exporter and --exporter-opt for custom values passed to the exporter. In the case of image exporter an image name can be specified.

For example:

go run ./examples/buildkit2/buildkit.go | buildctl build --exporter image --exporter-opt name=docker.io/moby/buildkit:dev

Accessing from ctr/dist:

ctr --namespace buildkit images ls
ctr --namespace buildkit rootfs unpack <manifest-sha>
ctr --namespace buildkit run -t docker.io/moby/buildkit:dev id ash

Local source

Buildkit now supports building from local sources. Snapshot of the local source files is created similarly to docker build build context. The implementation is based on the incremental context send feature in docker-v17.07. To use in buildctl the source definition needs to define a name for local endpoint, and buildctl build command provides a mapping from this name to a local directory with a --local flag.

go run ./examples/buildkit3/buildkit.go --local  | buildctl build --local buildkit-src=.

Typed Dockerfile parsing

Didn't manage to merge this PR yet. Still in code-review.

Feedback for `RUN --mount` / `COPY --chown`

There was some new discussion around RUN --mount or COPY --chown feature. Currently, it seems that it may be best to try the shared cache capabilities described in RUN --mount in https://github.com/moby/buildkit first(it already supports the generic mounting capabilities). So to unblock the people waiting only on the file owner change features it may make sense to implement COPY --chown first. Another related candidate for v17.08 release is https://github.com/moby/moby/issues/32816.