Use ARG in stage name

[Bidski]

Is it possible to do something like this?

ARG variant

FROM archlinux:latest AS base-image
RUN some things

FROM base-image AS base-image-${variant}
RUN some variant specific things

FROM base-image-${variant} AS base-image-${variant}-sub-image-1
RUN some more things

FROM base-image-${variant} AS base-image-${variant}-sub-image-2
RUN some more things

I have seen recommendation on to use

ARG variant 

FROM archlinux:latest AS base-image
RUN some things

FROM base-image AS base-image-generic
RUN some variant specific things

FROM base-image AS base-image-not-generic
RUN some variant specific things

FROM base-image-${variant}
RUN some more things

However I have quite a few sub-images and at least 3 variants. So I would have to duplicate every single stage at least 3 times in order to make this work. This is extremely undesirable as it is just duplicating code and making maintaining it almost impossible.

[tonistiigi]

I don't quite understand how this helps you. An arg can only have one single value for the whole build, so essentially the value in the stage name is a constant.

You never need to duplicate code when you are making more cases like this. You only need to add new FROM statements that point to already defined stage. A stage can be just the FROM command.

For the way how to avoid defining FROM for each condition if they are the same we will probably go with #2389 #1772 .

[Bidski]

I guess this might be a misunderstanding on my part with how stages work then. Does each stage effectively create an image? (as in if I run docker image ls after building everything will all of the stages be listed there?)

If so, then the arg will result in the creation of different images with different names and will prevent incompatible images being used as a cache in the build process.

[tonistiigi]

Does each stage effectively create an image?

No. Each build creates one result (that can be an image). Also, a stage name is not an image name.

Combining is done in higher-level tools, eg. https://github.com/docker/buildx/blob/master/docs/reference/buildx_bake.md

[Bidski]

How does caching work on stages then?

[tonistiigi]

How does caching work on stages then?

The issue tracker is not a good place for a broad question like this.

[Bidski]

Where is a suitable place?

[tonistiigi]

Stackoverflow, github discussions, #buildkit on slack

[thaJeztah]

This came up recently while I was reviewing a pull-request in moby/moby#44513 (comment), and I have a use-case for this (combined with #3352).

In the moby repository we have some build-stages that should only be executed on specific architectures. For all other architectures, the stage should be a "no-op". To implement that, we're adding "stub" / "dummy" stages for each of those architectures; simplified, this looks like:

# dummy stages for unsupported platforms
FROM scratch AS build-windows-amd64
FROM scratch AS build-windows-arm64
FROM scratch AS build-linux-arm
FROM scratch AS build-linux-amd64
FROM scratch AS build-linux-ppc64le
FROM scratch AS build-linux-i386
FROM scratch AS build-linux-arm64
FROM scratch AS build-linux-riscv

# only supported on linux/amd64
FROM alpine AS build-linux-amd64
RUN echo "do some actual stuff"

FROM build-${TARGETOS}-${TARGETARCH} AS build

# ....
# ....
# ....

FROM baseimage AS final
COPY --from=build /. /bin/

As can be seen, this leads to a lot of boilerplating / repetition, and requires a dummy for each possible os/arch combination that isn't supported, which is error-prone.

If AS would allow for an ARG to be used, we can (combined with #3352, which looks to be supported) use the TARGETOS and TARGETARCH options to automatically create a dummy stage for "whichever" platform is used as target;

# dummy stage for any unsupported platform
FROM scratch AS build-${TARGETOS}-${TARGETARCH}

# only supported on linux/amd64
FROM alpine AS build-linux-amd64
RUN echo "do some actual stuff"

FROM build-${TARGETOS}-${TARGETARCH} AS build

# ....
# ....
# ....

FROM baseimage AS final
COPY --from=build /. /bin/