build system: simplify docker image pinning #20877
Conversation
github-actions
bot
added
Area: build system
|
@pakic0o: Would you mind testing this? |
maribu
force-pushed
the
tools/docker
branch
from
6803cec to
8e31094
Compare
maribu
added
the
Type: bug
|
I'd like to test on macOS before it's merged, is that possible? |
|
Yeah sure, we can wait. (I also tested on MacOS on ARM64) |
Teufelchen1
dismissed
their stale review
Affected user wants to do further testing; Waiting for it.
|
Sorry for my previous message, I wasn't on the right branch... It seems to work but now I have a different issue, which doesn't seem related to this PR % BUILD_IN_DOCKER=1 BOARD=samd21-xpro make -C examples/hello-world
make: Entering directory '/Users/franciscoacosta/git/RIOT-OS/RIOT/examples/hello-world'
Launching build container using image "docker.io/riot/riotbuild@sha256:beb980f9ef00e7143fba913f3cb927866f987fe570141e145539c42141a940bd".
docker run --rm --tty --user $(id -u) -v '/usr/share/zoneinfo.default/Europe/Berlin:/etc/localtime:ro' -v '/Users/franciscoacosta/git/RIOT-OS/RIOT:/data/riotbuild/riotbase:delegated' -v '/Users/franciscoacosta/.cargo/registry:/data/riotbuild/.cargo/registry:delegated' -v '/Users/franciscoacosta/.cargo/git:/data/riotbuild/.cargo/git:delegated' -e 'RIOTBASE=/data/riotbuild/riotbase' -e 'CCACHE_BASEDIR=/data/riotbuild/riotbase' -e 'BUILD_DIR=/data/riotbuild/riotbase/build' -e 'BUILD_IN_DOCKER=/data/riotbuild/riotbase/examples/hello-world/1' -e 'RIOTPROJECT=/data/riotbuild/riotbase' -e 'RIOTCPU=/data/riotbuild/riotbase/cpu' -e 'RIOTBOARD=/data/riotbuild/riotbase/boards' -e 'RIOTMAKE=/data/riotbuild/riotbase/makefiles' -e 'BOARD=samd21-xpro' -e 'DISABLE_MODULE=' -e 'DEFAULT_MODULE=' -e 'FEATURES_REQUIRED=' -e 'FEATURES_BLACKLIST=' -e 'FEATURES_OPTIONAL=' -e 'USEMODULE=' -e 'USEPKG=' -w '/data/riotbuild/riotbase/examples/hello-world/' 'docker.io/riot/riotbuild@sha256:beb980f9ef00e7143fba913f3cb927866f987fe570141e145539c42141a940bd' make
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
docker: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting "/usr/share/zoneinfo.default/Europe/Berlin" to rootfs at "/etc/localtime": mount /usr/share/zoneinfo.default/Europe/Berlin:/etc/localtime (via /proc/self/fd/7), flags: 0x5000: not a directory: unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type.
make: *** [/Users/franciscoacosta/git/RIOT-OS/RIOT/makefiles/docker.inc.mk:369: ..in-docker-container] Error 125
make: Leaving directory '/Users/franciscoacosta/git/RIOT-OS/RIOT/examples/hello-world' |
Which message do you mean? The issue report in #20853 altogether, or just some part of it? |
|
I deleted it, I've first said that it didn't work but I had tested it in the master branch instead of this PR. With this PR I've got the result I've posted, so it seems to work but I have to figure out probably my local docker settings which may be the cause why it doesn't start to build |
There was a problem hiding this comment.
One minor nit below.
Otherwise I can confirm that the changes of this PR work fine both with docker and podman on my 64-bit Linux machine.
I just wondered why we didn't go that easier route in the first place. Looking back at the discussion in #20472 it seems we just tried with the local hash and not the remote one.
|
@pakic0o Could you give this another spin? I think with adding the |
mguetschow
added
the
CI: ready for build
|
Looks like this now needs a rebase after #20888 |
It turns out that the ID mechanics of docker are even more crazy than realized before: On Linux (x86_64) they use a different SHA256 when referring to a locally installed image than when referring to the same image at dockerhub. On Mac OS (Apple Silicon), the use the repo SHA256 also when referring to the local image. Instead of increasing the complexity of the current solution even more by covering both cases, we now use `docker.io/riot/riotbuild@sha256:<SHA256_OF_DOCKERHUB_IMAGE>` to refer to a specific docker image, which hopefully works across systems. Instead of pulling the image explicitly, we now can rely on docker to do so automatically if the pinned image is not found locally. As a result, the knob to disable automatic pulling has been dropped. Fixes RIOT-OS#20853
|
@pakic0o Any chance you could give this another spin? |
|
With the confirmation this fixes the issue in #20853 (comment), let's get this in :) |
|
I still have the same issue but I think it's related to my docker configuration 😢 but yeah, let's get it in! Thanks for the fix!! 🙏 |
Contribution description
It turns out that the ID mechanics of docker are even more crazy than realized before: On Linux (x86_64) they use a different SHA256 when referring to a locally installed image than when referring to the same image at dockerhub. On Mac OS (Apple Silicon), the use the repo SHA256 also when referring to the local image.
Instead of increasing the complexity of the current solution even more by covering both cases, we now use
docker.io/riot/riotbuild@sha256:<SHA256_OF_DOCKERHUB_IMAGE>to refer to a specific docker image, which hopefully works across systems.Instead of pulling the image explicitly, we now can rely on docker to do so automatically if the pinned image is not found locally. As a result, the knob to disable automatic pulling has been dropped.
Testing procedure
make BUILD_IN_DOCKER=1 -C examples/defaultshould now work on Mac OS on Apple Silicon as well as on Linux machines. It should automatically pull the container, if not locally present.Issues/PRs references
Fixes #20853