build: support for named contexts(stages)

[tonistiigi]

This PR enables setting named Dockerfile contexts moby/buildkit#2521 from buildx.

There were some concerns over the flag --context with alternatives --build-context and --stage proposed. This has been updated to --stage.

buildx build --stage alpine=docker-image://alpine:1.12 .
buildx build --stage project2=path/to/source .
buildx build --stage buildkit-src=https://github.com/moby/buildkit .

I'll implement bake support separately. Eventually, it should be possible to set bake targets also this way.

@AkihiroSuda @crazy-max

[thaJeztah]

There were some concerns over the flag --context with alternatives --build-context and --stage proposed.

Curious; where was that discussion? What were the main concerns? I can understand (build-) context being somewhat overloaded (with the later introduction of docker context, and if the source is not a (local) path); that said; we invested a lot in that term over the Years.

I'm not a fan of --stage - it feels a bit odd. I would consider something a stage if it's a (range of steps) in a Dockerfile, but naming "anything that can be used as source" a stage feels a bit like a stretch; these stages can also not be used as --target, so I'm not sure we should conflate those terms.

If we would be looking for alternatives; would --input or --source work? That way we could call them (named) sources or (named) inputs. (--input would make it somewhat symmetrical with --output, which may be a nice touch).

[thaJeztah]

Trying to get my head around the docker-image example; does that mean that I can randomly override images without the Dockerfile having an indication that I can? So the following will build from a centos:7 image?

docker buildx build --stage ubuntu=docker-image://centos:7 -<<EOF
FROM ubuntu
EOF

And what happens in (e.g.);

docker buildx build --stage ubuntu=docker-image://centos -<<EOF
ARG IMAGE_TAG=20.04
FROM ubuntu:$IMAGE_TAG
EOF

and:

docker buildx build --stage ubuntu=docker-image://centos:7 -<<EOF
FROM ubuntu:20.04 AS ubuntu

FROM ubuntu
EOF

Should it print a warning or error?

[crazy-max]

Trying to get my head around the docker-image example; does that mean that I can randomly override images without the Dockerfile having an indication that I can? So the following will build from a centos:7 image?

@thaJeztah It only concerns named contexts iiuc.

[crazy-max]

I tried the following use case:

# frontend
$ docker buildx build --push \
  --build-arg "CHANNEL=mainline" \
  --build-arg "BUILDKIT_CONTEXT_KEEP_GIT_DIR=1" \
  --tag "crazymax/dockerfile:edge" \
  --file "./frontend/dockerfile/cmd/dockerfile-frontend/Dockerfile" \
  https://github.com/moby/buildkit.git#master

# buildkit
$ docker buildx build --load \
  --build-arg "BUILDKIT_CONTEXT_KEEP_GIT_DIR=1" \
  --tag=moby/buildkit:local \
  https://github.com/moby/buildkit.git#master

# buildx
$ docker buildx bake https://github.com/tonistiigi/buildx.git#named-contexts binaries
$ cp ./bin/buildx ~/.docker/cli-plugins/docker-buildx

# create builder
$ docker buildx create --name test-context --driver-opt image=moby/buildkit:local --use

# syntax=crazymax/dockerfile:edge
FROM ubuntu:20.04 AS ubuntu-base
RUN echo "hello" > /hello

FROM ubuntu
COPY --from=ubuntu-base /etc/os-release /os-release
RUN cat /os-release
COPY --from=ubuntu-base /hello /hello
RUN cat /hello

$ docker buildx build --no-cache --stage ubuntu-base=docker-image://centos:7 .
WARN[0000] No output specified for docker-container driver. Build result will only remain in the build cache. To push result image into registry use --push or to load image into docker use --load 
#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 270B 0.0s done
#1 DONE 0.1s

#2 [internal] load .dockerignore
#2 transferring context: 2B done
#2 DONE 0.1s

#3 resolve image config for docker.io/crazymax/dockerfile:edge
#3 ...

#4 [auth] crazymax/dockerfile:pull token for registry-1.docker.io
#4 DONE 0.0s

#3 resolve image config for docker.io/crazymax/dockerfile:edge
#3 DONE 1.4s

#5 docker-image://docker.io/crazymax/dockerfile:edge@sha256:40b97b20835408da69b79e6f7a5ec5a4d2dbb6fc1b73b1457ecff9a09ac2de2e
#5 resolve docker.io/crazymax/dockerfile:edge@sha256:40b97b20835408da69b79e6f7a5ec5a4d2dbb6fc1b73b1457ecff9a09ac2de2e 0.0s done
#5 sha256:249717fc1bd58ee73ac4b4f84b9a371e132e91e21d43c627b1bf1e222d7196aa 0B / 9.87MB 0.1s
#5 sha256:249717fc1bd58ee73ac4b4f84b9a371e132e91e21d43c627b1bf1e222d7196aa 1.05MB / 9.87MB 0.3s
#5 sha256:249717fc1bd58ee73ac4b4f84b9a371e132e91e21d43c627b1bf1e222d7196aa 3.15MB / 9.87MB 0.4s
#5 sha256:249717fc1bd58ee73ac4b4f84b9a371e132e91e21d43c627b1bf1e222d7196aa 5.24MB / 9.87MB 0.6s
#5 sha256:249717fc1bd58ee73ac4b4f84b9a371e132e91e21d43c627b1bf1e222d7196aa 6.29MB / 9.87MB 0.8s
#5 sha256:249717fc1bd58ee73ac4b4f84b9a371e132e91e21d43c627b1bf1e222d7196aa 8.39MB / 9.87MB 0.9s
#5 sha256:249717fc1bd58ee73ac4b4f84b9a371e132e91e21d43c627b1bf1e222d7196aa 9.87MB / 9.87MB 1.0s done
#5 extracting sha256:249717fc1bd58ee73ac4b4f84b9a371e132e91e21d43c627b1bf1e222d7196aa
#5 extracting sha256:249717fc1bd58ee73ac4b4f84b9a371e132e91e21d43c627b1bf1e222d7196aa 0.2s done
#5 DONE 1.2s

#6 [internal] load metadata for docker.io/library/ubuntu:latest
#6 ...

#7 [auth] library/ubuntu:pull token for registry-1.docker.io
#7 DONE 0.0s

#6 [internal] load metadata for docker.io/library/ubuntu:latest
#6 DONE 0.9s

#9 [auth] library/centos:pull token for registry-1.docker.io
#9 DONE 0.0s

#8 resolve image config for docker.io/library/centos:7
#8 DONE 0.2s

#10 [stage-1 1/5] FROM docker.io/library/ubuntu@sha256:626ffe58f6e7566e00254b638eb7e0f3b11d4da9675088f4781a50ae288f3322
#10 resolve docker.io/library/ubuntu@sha256:626ffe58f6e7566e00254b638eb7e0f3b11d4da9675088f4781a50ae288f3322 0.0s done
#10 ...

#11 [context ubuntu-base] //centos:7
#11 resolve docker.io/library/centos:7 0.2s done
#11 DONE 0.0s

#10 [stage-1 1/5] FROM docker.io/library/ubuntu@sha256:626ffe58f6e7566e00254b638eb7e0f3b11d4da9675088f4781a50ae288f3322
#10 sha256:7b1a6ab2e44dbac178598dabe7cff59bd67233dba0b27e4fbd1f9d4b3c877a54 0B / 28.57MB 0.2s
#10 sha256:7b1a6ab2e44dbac178598dabe7cff59bd67233dba0b27e4fbd1f9d4b3c877a54 4.19MB / 28.57MB 0.6s
#10 sha256:7b1a6ab2e44dbac178598dabe7cff59bd67233dba0b27e4fbd1f9d4b3c877a54 6.29MB / 28.57MB 1.2s
#10 sha256:7b1a6ab2e44dbac178598dabe7cff59bd67233dba0b27e4fbd1f9d4b3c877a54 8.39MB / 28.57MB 2.3s
#10 sha256:7b1a6ab2e44dbac178598dabe7cff59bd67233dba0b27e4fbd1f9d4b3c877a54 10.49MB / 28.57MB 3.3s
#10 sha256:7b1a6ab2e44dbac178598dabe7cff59bd67233dba0b27e4fbd1f9d4b3c877a54 12.58MB / 28.57MB 5.0s
#10 sha256:7b1a6ab2e44dbac178598dabe7cff59bd67233dba0b27e4fbd1f9d4b3c877a54 14.68MB / 28.57MB 6.2s
#10 sha256:7b1a6ab2e44dbac178598dabe7cff59bd67233dba0b27e4fbd1f9d4b3c877a54 16.78MB / 28.57MB 7.1s
#10 sha256:7b1a6ab2e44dbac178598dabe7cff59bd67233dba0b27e4fbd1f9d4b3c877a54 18.87MB / 28.57MB 8.0s
#10 sha256:7b1a6ab2e44dbac178598dabe7cff59bd67233dba0b27e4fbd1f9d4b3c877a54 20.97MB / 28.57MB 8.6s
#10 sha256:7b1a6ab2e44dbac178598dabe7cff59bd67233dba0b27e4fbd1f9d4b3c877a54 24.12MB / 28.57MB 9.0s
#10 sha256:7b1a6ab2e44dbac178598dabe7cff59bd67233dba0b27e4fbd1f9d4b3c877a54 26.21MB / 28.57MB 9.5s
#10 sha256:7b1a6ab2e44dbac178598dabe7cff59bd67233dba0b27e4fbd1f9d4b3c877a54 28.57MB / 28.57MB 9.8s done
#10 extracting sha256:7b1a6ab2e44dbac178598dabe7cff59bd67233dba0b27e4fbd1f9d4b3c877a54
#10 ...

#11 [context ubuntu-base] //centos:7
#11 sha256:2d473b07cdd5f0912cd6f1a703352c82b512407db6b05b43f2553732b55df3bc 76.10MB / 76.10MB 9.4s done
#11 extracting sha256:2d473b07cdd5f0912cd6f1a703352c82b512407db6b05b43f2553732b55df3bc
#11 ...

#10 [stage-1 1/5] FROM docker.io/library/ubuntu@sha256:626ffe58f6e7566e00254b638eb7e0f3b11d4da9675088f4781a50ae288f3322
#10 extracting sha256:7b1a6ab2e44dbac178598dabe7cff59bd67233dba0b27e4fbd1f9d4b3c877a54 0.8s done
#10 DONE 10.6s

#11 [context ubuntu-base] //centos:7
#11 extracting sha256:2d473b07cdd5f0912cd6f1a703352c82b512407db6b05b43f2553732b55df3bc 2.3s done
#11 DONE 11.7s

#12 [stage-1 2/5] COPY --from=ubuntu-base /etc/os-release /os-release
#12 CACHED

#13 [stage-1 3/5] RUN cat /os-release
#13 CACHED

#14 [stage-1 4/5] COPY --from=ubuntu-base /hello /hello
#14 ERROR: failed to calculate checksum of ref w5321atmoskkk1m1sbds778tk::i1rd3owef5do8uz8tbinfqdxs: "/hello": not found

#12 [stage-1 2/5] COPY --from=ubuntu-base /etc/os-release /os-release
#12 DONE 0.5s
------
 > [stage-1 4/5] COPY --from=ubuntu-base /hello /hello:
------
Dockerfile:8
--------------------
   6 |     COPY --from=ubuntu-base /etc/os-release /os-release
   7 |     RUN cat /os-release
   8 | >>> COPY --from=ubuntu-base /hello /hello
   9 |     RUN cat /hello
  10 |
--------------------
error: failed to solve: failed to compute cache key: failed to calculate checksum of ref w5321atmoskkk1m1sbds778tk::i1rd3owef5do8uz8tbinfqdxs: "/hello": not found

I would expect the RUN echo "hello" > /hello to be called but doesn't seem like it.

Also found smth odd that might not be linked but I'm not able to prune cache for:

#12 [stage-1 2/5] COPY --from=ubuntu-base /etc/os-release /os-release
#12 CACHED

#13 [stage-1 3/5] RUN cat /os-release
#13 CACHED

Even with docker buildx build --no-cache

[tonistiigi]

What were the main concerns?

The issue was with smth like docker --context=dockerctx buildx build --context foo=bar .. I agree --stage is not ideal as it is too Dockerfile specific while this can be easily supported by any frontend(eg. like --target).

I don't have good alternatives. I don't like --input and --source as that is not really the inputs, and --source is only one use case of the flag. The concept is more like a --define.

Trying to get my head around the docker-image example; does that mean that I can randomly override images without the Dockerfile having an indication that I can? Should it print a warning or error?

Yes, the caller has the ultimate control to configure the build with the cli flags. No warning/error. It is the same behavior as

ARG UBUNTU
FROM $UBUNTU

So the following will build from a centos:7 image?

1. yes
2. no (ubuntu:latest not used)
3. yes

I would expect the RUN echo "hello" > /hello to be called but doesn't seem like it.

With --stage ubuntu-base= you are overwriting the definition of that name. So the contents need to contain /hello file. If you wish to overwrite a base of that stage you would do --stage ubuntu:20.04=foo or define your stage like:

FROM ubuntu:20.04 AS ubuntu-base
FROM ubuntu-base AS ubuntu-build
RUN ...

FROM
COPY --from=ubuntu-build

For the cache comment, make a separate reproducer with clean buildx create cycle. I'm not sure if it is related.

[crazy-max]

With --stage ubuntu-base= you are overwriting the definition of that name. So the contents need to contain /hello file. If you wish to overwrite a base of that stage you would do --stage ubuntu:20.04=foo or define your stage like

Ok LGTM.

For the cache comment, make a separate reproducer with clean buildx create cycle. I'm not sure if it is related.

I'm not reproducing anymore. Maybe smth off with my nodes.

[thaJeztah]

(From slack) --build-context sounds like a good alternative to me

[thaJeztah]

Nice!

FWIW; one of the discussions in the maintainers meeting brought to light that the additional build-contexts currently do not use an "interactive session" to send the build-context. So if a named context is specified, but the Dockerfile only consumes a single file from that context, the whole context would still be sent to the daemon.

So in the example below, the whole path/to/project/source directory would be sent, and not just myfile.

docker buildx build --build-context project=path/to/project/source .

 FROM alpine
 COPY --from=project myfile /

I think the "interactive session" for sending the build context is one of the biggest benefits of using BuildKit instead of the classic builder; @tonistiigi mentioned that it would be possible to fix this, but while (if) that's not done yet, I think we should mention this limitation somewhere in the documentation.

Not sure if there's a tracking issue for this btw (happy to create one if there's no ticket yet).