Running docker container / building image on M1 Mac (How-to and troubleshooting)

[sharbel97]

Hi everyone, I've spent a number of hours trying to run valhalla using docker. I'm new to docker and with this project this is the first time I tried to learn docker thoroughly. I believe this discussion will help to lower the initial barrier to first time contributors using M1 Mac.

Few issues, and I'm having trouble differentiating between the 2 images valhalla has.

Finding (1): Both images when pulled from docker hub, exit immediately upon running due to (I believe) mismatch platform architecture. I tried pulling (and running) with --platform linux/arm64 but nothing works here. Same result. How I pulled/ran is attached at the end of this description. Container logs are empty. Docker is enabled to emulate.

Finding (2): Next step I tried is building image with the Dockerfile given. The native valhalla image builds fine, and runs with no errors, but every curl request I tried returns curl: (52) Empty reply from server. How I built/ran is attached at the end of this description. Container logs are empty.

The gis-ops image however doesn't build due to an error (I tried pip install, didn't work):

~/code/docker-valhalla$ docker build -t ghcr.io/gis-ops/docker-valhalla/valhalla:latest --platform linux/arm64 .
[+] Building 1.6s (16/16) FINISHED                                                                                                                                
 => [internal] load build definition from Dockerfile                                                                                                         0.0s
 => => transferring dockerfile: 3.27kB                                                                                                                       0.0s
 => [internal] load .dockerignore                                                                                                                            0.0s
 => => transferring context: 98B                                                                                                                             0.0s
 => [internal] load metadata for docker.io/library/ubuntu:22.04                                                                                              1.2s
 => [internal] load metadata for ghcr.io/valhalla/valhalla:latest                                                                                            0.6s
 => [auth] library/ubuntu:pull token for registry-1.docker.io                                                                                                0.0s
 => [runner_base 1/8] FROM docker.io/library/ubuntu:22.04@sha256:67211c14fa74f070d27cc59d69a7fa9aeff8e28ea118ef3babc295a0428a6d21                            0.0s
 => [builder 1/2] FROM ghcr.io/valhalla/valhalla:latest@sha256:e59a8c3f2cf6a814c6d33c971c9c3e263c7193132575cf3c5ef21eaf91827eb1                              0.0s
 => [internal] load build context                                                                                                                            0.2s
 => => transferring context: 13.95kB                                                                                                                         0.1s
 => CACHED [runner_base 2/8] RUN apt-get update > /dev/null &&     export DEBIAN_FRONTEND=noninteractive &&     apt-get install -y libluajit-5.1-2       li  0.0s
 => CACHED [builder 2/2] RUN cd /usr/local/bin &&   preserve="valhalla_service valhalla_build_tiles valhalla_build_config valhalla_build_admins valhalla_bu  0.0s
 => CACHED [runner_base 3/8] COPY --from=builder /usr/local /usr/local                                                                                       0.0s
 => CACHED [runner_base 4/8] COPY --from=builder /usr/lib/python3/dist-packages/valhalla/* /usr/lib/python3/dist-packages/valhalla/                          0.0s
 => CACHED [runner_base 5/8] RUN groupadd -g 59999 valhalla &&   useradd -lmu 59999 -g valhalla valhalla &&   mkdir /custom_files &&   if [ 59999 != 59999   0.0s
 => CACHED [runner_base 6/8] COPY scripts/. /valhalla/scripts                                                                                                0.0s
 => CACHED [runner_base 7/8] WORKDIR /custom_files                                                                                                           0.0s
 => ERROR [runner_base 8/8] RUN python -c "import valhalla,sys; print (sys.version, valhalla)"     && valhalla_build_config | jq type     && cat /usr/local  0.2s
------                                                                                                                                                            
 > [runner_base 8/8] RUN python -c "import valhalla,sys; print (sys.version, valhalla)"     && valhalla_build_config | jq type     && cat /usr/local/src/valhalla_version     && valhalla_build_tiles -v     && ls -la /usr/local/bin/valhalla*:
#16 0.191 Traceback (most recent call last):
#16 0.191   File "/usr/lib/python3/dist-packages/valhalla/__init__.py", line 2, in <module>
#16 0.191     from .python_valhalla import *
#16 0.191 ModuleNotFoundError: No module named 'valhalla.python_valhalla'
#16 0.191 
#16 0.191 During handling of the above exception, another exception occurred:
#16 0.191 
#16 0.191 Traceback (most recent call last):
#16 0.191   File "<string>", line 1, in <module>
#16 0.191   File "/usr/lib/python3/dist-packages/valhalla/__init__.py", line 4, in <module>
#16 0.191     from python_valhalla import *
#16 0.191 ModuleNotFoundError: No module named 'python_valhalla'
------
executor failed running [/bin/sh -c python -c "import valhalla,sys; print (sys.version, valhalla)"     && valhalla_build_config | jq type     && cat /usr/local/src/valhalla_version     && valhalla_build_tiles -v     && ls -la /usr/local/bin/valhalla*]: exit code: 1

How I built (same for both images)
in docker-valhalla: docker build -t ghcr.io/gis-ops/docker-valhalla/valhalla:latest . [errors when building only, not pulling]
in valhalla: docker build -t valhalla/valhalla:latest . [success]
With and without the --platform flag attempted, same result.

How I ran the images, assume I already downloaded the tiles (pbf) in custom_files
gis-ops image docker run -dt --name valhalla -p 8002:8002 -v $PWD/custom_files:/custom_files ghcr.io/gis-ops/docker-valhalla/valhalla:latest
native image docker run -dt --name valhalla -p 8002:8002 -v $PWD/custom_files:/custom_files valhalla/valhalla:latest
At no point in the whole process do I get messages saying graph is being built. The tiles are in fact mounted inside the container when I run it, checked via docker dashboard.

How I curl'd
I tried the same curl instructions from docker-valhalla/tests/test.sh there is a curl instruction there appropriate for the Andorra tiles.
curl -s -XPOST 'http://localhost:8002/route' -H'Content-Type: application/json' --data-raw '{"locations":[{"lat": 42.546504,"lon": 1.591129},{"lat": 42.507667,"lon": 1.542721}],"costing": "auto"}'

and this one from docker-compose
curl http://localhost:8002/status

I believe I've tried all the options offered in README and other discussions in this repo the valhalla one. Any help or feedback is appreciated.

[nilsnolde]

Couple of thoughts:

• the images are not interchangeable, ours is based on the upstream one which doesn’t have any automations
• that also means that you can’t just attempt to build ours for arm without first building the upstream Valhalla image for arm and using that as the base

[sharbel97]

Ok, that solves my problem. I built the upstream image with name valhalla/valhalla but the gis-ops image was trying to base from ghcr.io/valhalla/valhalla which didn't exist . Soon as I rebuilt the upstream image with the sought name, it worked.

The curl requests now work, as well as the test script from the docker-valhalla readme. Thank you! @nilsnolde

[nilsnolde]

Ok great:) did I understand right, that our gis-ops image works for ARM as well after you built the upstream Valhalla image locally for ARM as ghcr.io/valhalla/valhalla? Or you’re just using the upstream one now? Did you have to adjust anything else, like the python stuff?

[sharbel97]

did I understand right, that our gis-ops image works for ARM as well after you built the upstream Valhalla image locally for ARM as ghcr.io/valhalla/valhalla

Correct

Or you’re just using the upstream one now

I used the gis-ops image

Did you have to adjust anything else, like the python stuff

No other adjustments made. Only thing outside of the README I used is the --platform option for building.

[amirzandi]

@sharbel97 thanks for sharing the experience. Could you please also share a step by step instruction from the scratch?