Missing or partial support for pattern substition in variable references (e.g. in RUN)

[andraxin]

Here is a minimal working example that reproduces this issue with across different Kaniko versions, with and without caching. (The script assumes bash and a terminal capable of highlighting).

cd "$(mktemp -d)"
{
    echo FROM alpine
    echo RUN var=a-b\; echo \${var/-//}
} | tee Dockerfile
e=$'\x1b' w=/workspace k=gcr.io/kaniko-project/executor
for i in {17..21}
do for c in "" "--cache --cache-repo -"
do echo "$e[33;1m$i:${c:2:6}$e[0m"
    docker run --rm -it -v "$PWD":$w $k:v0.$i.0 --no-push $c |
        sed "s/.*missing.*/$e[31;1m&$e[0m/"
    done
done
rm -vfr "$PWD"
cd -

The output is omitted for brevity (just run the script). The results are mixed. The following table provides a summary.

version	cache	no-cache
v0.17.0	OK	OK
v0.18.0	OK	OK
v0.19.0	KO	KO
v0.20.0	OK	KO
v0.21.0	OK	KO

Essentially, v0.19.0 is broken irrespective of caching, while earlier versions appear to be fine, and later versions only break when caching is turned on.

The last and likely most relevant part of the compounded error comes from this line

[zaventh]

Still an issue in v1.2.0. For example, the parameter expansion "${buildArch##*-}" works in Docker build, but not in Kaniko.

[nirav-chotai]

I cannot build the airflow image using kaniko, it works fine with the docker build.

Error

error building image: error building stage: failed to optimize instructions: failed to process "RUN mkdir -pv /usr/share/man/man1     && mkdir -pv /usr/share/man/man7     && export ${ADDITIONAL_DEV_APT_ENV?}     && bash -o pipefail -e -u -x -c \"${DEV_APT_COMMAND}\"     && bash -o pipefail -e -u -x -c \"${ADDITIONAL_DEV_APT_COMMAND}\"     && apt-get update     && apt-get install -y --no-install-recommends            ${DEV_APT_DEPS}            ${ADDITIONAL_DEV_APT_DEPS}     && apt-get autoremove -yqq --purge     && apt-get clean     && rm -rf /var/lib/apt/lists/*": missing ':' in substitution

Code

...
RUN mkdir -pv /usr/share/man/man1 \
    && mkdir -pv /usr/share/man/man7 \
    && export ${ADDITIONAL_DEV_APT_ENV?} \
    && bash -o pipefail -e -u -x -c "${DEV_APT_COMMAND}" \
    && bash -o pipefail -e -u -x -c "${ADDITIONAL_DEV_APT_COMMAND}" \
    && apt-get update \
    && apt-get install -y --no-install-recommends \
           ${DEV_APT_DEPS} \
           ${ADDITIONAL_DEV_APT_DEPS} \
    && apt-get autoremove -yqq --purge \
    && apt-get clean \
    && rm -rf /var/lib/apt/lists/*
...

[abprohorov]

Same on v1.3.0 with cache

[euven]

Still seeing this on the latest version (1.3.0) - removing the --cache flag makes things work.

[darkdragon-001]

Same problem in latest version using cache:

Shell form:

SHELL ["/bin/bash", "-c"]
RUN wget -q -O - https://dl.bintray.com/boostorg/release/${BOOST_VERSION}/source/boost_${BOOST_VERSION//./_}.tar.bz2 | tar xj

Exec form:

RUN ["/bin/bash", "-c", "wget -q -O - https://dl.bintray.com/boostorg/release/${BOOST_VERSION}/source/boost_${BOOST_VERSION//./_}.tar.bz2 | tar xj"]

[darkdragon-001]

Checking Dockerfile documentation, environment variable substitution does only happen for a list of commands not including RUN since it is executed in a container with environment variables set.

[markus-geiger]

I also ran int this issue with:

RUN something --version="release-${APP_VERSION%.*}"

A workaround with

ARG APP_VERSION_MAJMIN=${APP_VERSION%.*}

just gave me: missing ':' in substitution. The SHELL used was ["bash"]

It would be nyce to fix it.

[marcinz]

I am seeing the same issue. I am having to move out all complex shell parameter expansions out of the docker file and plug them in through build args. These expansions should be handled by the shell in a shell version of the RUN command, but Kaniko cache seems to be unaware of that.

[guillaume-d]

I am seeing the same issue. I am having to move out all complex shell parameter expansions out of the docker file and plug them in through build args. These expansions should be handled by the shell in a shell version of the RUN command, but Kaniko cache seems to be unaware of that.

A more local (but still pretty lame...) workaround may be to reformulate the expansions using command substitution and a pipeline (as in $(echo "$buildArch" | grep -o '[^-]*$') instead of ${buildArch##*-}), which Kaniko at least seems not to choke on.

[Leletir]

Still an error in 1.6.0 with cache

[cemoktra]

Please fix this bug. Its annoying as hell and makes it unusable

[michaellmonaghan]

This issue is still present when building with cache on gcr.io/kaniko-project/executor:latest 1c812ffa8ec1.

[MatanAmoyal]

Still happen on 1.7.0 kaniko version with cache.
Any plan to fix it?

[tushar-door]

Is there a workaround for this? Like, not using cache?

[cemoktra]

@tushar-door you can work around by moving the problematic lines to a shell script

[ksa-real]

Another workaround: something $(sh -c 'echo ${APP_VERSION%-*}') instead of something ${APP_VERSION%-*}. Pretty annoying...

[PapaNappa]

Another workaround: something $(sh -c 'echo ${APP_VERSION%-*}') instead of something ${APP_VERSION%-*}. Pretty annoying...

That’s the best workaround for me.
Interestingly, RUN echo $(sh -c 'echo ${FOO#foo}') works, but RUN echo "$(sh -c 'echo ${FOO#foo}')" produces the same error message.
It seems when inside ", kaniko always tries to substitute the variable, even when it’s actually nested inside $().

(Edit: fixed quotes)

[ksa-real]

FYI you are missing closing ' in both cases

[waddles]

I found the easiest and most obvious workaround is to do the substitution with an if. eg.

-RUN curl --location --output /envsubst "https://github.com/a8m/envsubst/releases/download/v${ENVSUBST_VERSION}/envsubst-${TARGETOS}-${TARGETARCH/amd64/x86_64}" && \
+RUN if [ "$TARGETARCH" = "amd64" ]; then TARGETARCH=x86_64; fi && \
+    curl --location --output /envsubst "https://github.com/a8m/envsubst/releases/download/v${ENVSUBST_VERSION}/envsubst-${TARGETOS}-${TARGETARCH}" && \

[kt315]

Hi all!
I found the promblem here.
Where cache is use we need get checksum before execute RUN instruction. For that need interpolate all variable of shell (ARG or ENV in Dockerfile can be new and we have to consider this). Lex shell library which used for this case support only syntax: $var or ${var} or ${var:(+|-|?)word}. Any other bash syntax will return "missing ':' in substitution"
Now can use only half sh syntax.

[mcarbonneaux]

A more local (but still pretty lame...) workaround may be to reformulate the expansions using command substitution and a pipeline (as in $(echo "$buildArch" | grep -o '[^-]*$') instead of ${buildArch##*-}), which Kaniko at least seems not to choke on.

i used the same type of workaround....

ex: "$(echo "${f}" | sed "s/.example$//")" in place of 'echo ${f%.example}'

[codethief]

I'm experiencing a very similar issue (the error message is slightly different). The following line in my Dockerfile

RUN VERSION_FROM_PACKAGE_JSON="$(grep --extended-regexp '"playwright": "([0-9].+)"' package-lock.json | cut -d ':' -f 2 | cut -d '"' -f 2)" \
  bash -c 'npm install "playwright@${VERSION_FROM_PACKAGE_JSON}"'

causes Kaniko to output

error building image: error building stage: failed to optimize instructions: failed to process "RUN VERSION_FROM_PACKAGE_JSON=\"$(grep --extended-regexp '\"playwright\": \"([0-9].+)\"' package-lock.json | cut -d ':' -f 2 | cut -d '\"' -f 2)\"   bash -c 'npm install \"playwright@${VERSION_FROM_PACKAGE_JSON}\"'": unexpected end of statement while looking for matching single-quote

If I disable the cache, the error no longer shows up – which, of course, is not a workaround but makes me believe the issue I'm seeing is related to the present discussion.

[kt315]

I have an idea. I'll try to fix it by the end of the week

[zaventh]

Good job @kt315 Thanks.

[domi27]

I can also confirm using Kaniko version 1.22 helped me out against this bug. Thank you @kt315