Development Report for July 17, 2017


Repo Proposal

Following features were added last week:

Git source

Source code from git repositories can now be accessed directly, similarly for images can be accessed, without the need to execute git clone. This has many performance and caching advantages. It accesses the remote repository using shallow fetches to only pull the required data and a uses a shared bare repository for intermediate cache between build invocations. The instruction cache for the git source is based on a commit hash and not string arguments. This means that you can always be sure that you are building the correct source and that you never build the same source twice.

Containerd exporter

Exporters are used for getting build artifacts out of buildkit. The first exporter that was implemented allows exposing the image to containerd so it can be run and pushed with ctr tool. buildctl has --exporter flag for specifying the exporter and --exporter-opt for custom values passed to the exporter. In the case of image exporter an image name can be specified.

For example:

go run ./examples/buildkit2/buildkit.go | buildctl build --exporter image --exporter-opt

Accessing from ctr/dist:

ctr --namespace buildkit images ls
ctr --namespace buildkit rootfs unpack <manifest-sha>
ctr --namespace buildkit run -t id ash

Local source

Buildkit now supports building from local sources. Snapshot of the local source files is created similarly to docker build build context. The implementation is based on the incremental context send feature in docker-v17.07. To use in buildctl the source definition needs to define a name for local endpoint, and buildctl build command provides a mapping from this name to a local directory with a --local flag.

go run ./examples/buildkit3/buildkit.go --local  | buildctl build --local buildkit-src=.

Typed Dockerfile parsing


Didn't manage to merge this PR yet. Still in code-review.

Feedback for RUN --mount / COPY --chown

There was some new discussion around RUN --mount or COPY --chown feature. Currently, it seems that it may be best to try the shared cache capabilities described in RUN --mount in first(it already supports the generic mounting capabilities). So to unblock the people waiting only on the file owner change features it may make sense to implement COPY --chown first. Another related candidate for v17.08 release is

Proposals for new Dockerfile features that need design feedback:

Add IMPORT/EXPORT commands to Dockerfile

Add DOCKEROS/DOCKERARCH default ARG to Dockerfile

Add support for RUN --mount

DAG image builder

Option to export the hash of the build context (new)

Allow --cache-from=* (new)

Provide advanced .dockeringore use-cases 2

New: RFC: Distributed BuildKit

If you are interested in implementing any of them, leave a comment on the specific issues.

Builder features currently in code-review:

Fix shallow git clone in docker-build


Build secrets has not got much traction. If you want this feature to become a reality, please make yourself heard.