ARM-64 build #216
Comments
|
Note: If I build the image from this repository, I can successfully build on ARM-64. This means that the current Dockerfile(s) are already ARM-64 friendly, it's just a matter of uploading the built image to docker hub. |
|
I believe there is already an open issue around this, but it hasn't seen any activity for a few months: The state of things has changed a bit since then... most notably, travis is no longer the primary CI for the the docker images. Due to changes in the Travis-CI offering, the CI system was ported over to Github Actions. While Github Actions runners are x86 only, I believe there are actions that have been developed by the community to use qemu or other hypervisors to enable building the arm images using x86 platforms, but I have yet to look into it. I believe the issue I referenced above mentions raspberry pi explicitly, but building out this repo to work on multiple arm platforms (windows, apple) should probably be the goal. The docker-postgis repo was initially build on top of the official postgres repo, but I have yet to look at how the official postgres repo is doing their arm builds either. Probably the next logical step should checking out the official postgres repo and seeing how they're doing arm builds, and investigating how to make Github Actions build arm images. |
|
@derdaele also, are you actually running docker on your Apple Silicon machine? If so, I'd like to know more about how this is accomplished. Thanks! |
|
Yes, I'm running the developer preview [1]. I tried to look into how the official postgres image is built and it looks like it's using the brewbash project, I'm not sure how easy it would be to integrate this here. However I stumbled upon this [2] GitHub action that seems to be able to do multi-arch builds. I'll give it a try and send a PR if it works. |
|
Right, while we don't necessary need to integrate with brewbash, what we probably want to do is analyze things that prevent us from integrating in the future. They have set of patterns and guidelines for the official images (a bit lengthy actually) that we try to keep in mind as we're making changes to this repo. If the dev preview of docker desktop for M1 is capable of running buildx, then they're further along than I thought. Seeing the progress on this is very encouraging. I believe this github action from docker was also the one I saw for doing the multiplatform builds. I would say we don't necessarily have to use it, but only that it does exist so building multiplatform should be possible without having to setup any custom github runner. At the moment, there's only one github action flow in this repo which was directly ported from travis, and uses shell scripts that are invoked by github actions rather than using things that are tightly bound to github actions. This prevents us from committing too heavily to the CI product, and allows folks to run the same processes in local linux vms (or to some extent, a macos or windows environment) without the need for some local github actions emulation framework. It looks as if modifying things to get buildx in the mix is what the next task is to tackle multiplatform builds for this repo. If you'd like to take a stab at that, it would be much appreciated! |
|
I ran a few tests and results are actually not so great.
The ARM-64 build is only successful for the following combination:
I identified two error sources:
However the workflow is somewhat cleaner than the current setup (cf https://github.com/derdaele/docker-postgis/blob/master/.github/workflows/main.yml) as it doesn't require maintaining parts of the current Makefile. I'm happy to start a PR with this workflow and manually include in the matrix the combinations above to have some ARM build if you are good with that. |
|
@derdaele It's actually good news that ANY builds worked on a github actions runner! It indicates that building arm64 will be possible within the CI. Good work, and thanks! I think it's already been identified that arm64 packages on debian aren't available for some postgres+postgis combinations, but there's not much that can be done about that since that is handled upstream. I think we can simply exclude them from the build on a per-version-combination basis. As for alpine, if memory serves, there were reports of successful builds on alpine on raspberry pis (issue #144) but they may be outdated. And I can't remember if this was for the "master branch" builds, release builds, or some other combination. There may be some more work necessary to get it working. What would be ideal would be a prescribed (scriptable) way to install/configure qemu on an x86 ubuntu environment with buildx and any other necessary tooling to build and push arm images to dockerhub. This would allow allow us to script it on a VM, add it to the setup scripts for github actions, then write/modify github actions workflow(s) to build the images. But contributors would also be able to use the same scripts on a local environment to accomplish the same thing. This will help especially for people who want to contribute PRs without breaking the CI builds too much. So I guess my immediate question is: would you be willing to attempt to tackle a shell script method of accomplishing docker/buildx/qemu installation... basically duplicating what github actions does with the actions/setup and/or docker buildx setup actions? Integration with the Makefile or templates is not necessary, but just something that is runnable from a shell script. |
|
@derdaele unless you've already started messing with shell scripts, I'd say it's not actually necessary. There's a little more to it than I thought, so I won't ask you to bother with it. What I'm seeing now is that the github actions runners might be setup by default with a docker environment that is not sufficient for doing multiplatform builds, so some experimentation is needed to find a good way to get the environment setup. The github action that docker provides to do this seems to accomplish this, but it would be more beneficial to have the setup encapsulated in a script that people can use in their own VMs outside of the github actions environment. I'm in the process of working all this out now, so I won't ask you to duplicate the effort. When do get these multiplatform images building, i'll push to a test repo and let you know so you can test on the M1 platform. I signed up for the docker development program i hopes of getting access to their M1 builds, but I'm not guaranteed to get approved and get access, so I won't be able to test for myself. |
|
Hey @phillipross, I did not start working on the shell script already. SGTM, I'd be happy to give a try to the test images. |
|
I have been trying to troubleshoot the ARM64 builds failing in buildx/QEMU for 11-3.0-alpine (which just got changed to 3.1, but I don't think that matters), and I discovered something very interesting - it works perfectly when building natively on Amazon Graviton ( I wonder what the significance of this is, but I don't really know enough about the PostGIS build process to have a good guess yet. |
|
I'm seeing the same thing and I believe it's something to do with QEMU. Thanks for extra datapoint though. Thus far I've been trying with an M1 running ubuntu 20.04.01 in parallels and wasn't sure if it was an issue with the parallels (it's a preview build) or something else. As it stands, I'm able to build, test, and run natively, but trying to do cross-platform with QEMU continues breaking at lower levels that I'm not so familiar with. I've thought about trying to build QEMU from source and seeing if that makes a difference. |
|
Any updates on this issue? |
|
|
Unfortunately, saying "it's a QEMU bug so use a multi-arch image" without providing any details is not useful, considering that |
|
The comment for docker/for-mac#5122 by @stephen-turner assumes the context only concerns running the containers, but the more pressing issue we're discussing here is actually building the container images themselves. The qemu bug is what is preventing the multiarch images from easily being built within the existing CI/CD framework being used by this repo. This repo currently uses github actions to build the containers and deploy them to docker hub. Github actions currently only offers amd64 platforms to do builds, so building the arm64 images requires QEMU to build the arm64 images. It's possible to configure github actions to use remote runners on alternate platforms (such as AWS instances or self-hosted environments) but there's a lot of complexity involved in doing that in a robust way. I was hoping the qemu bug(s) would be resolved soonish or github ations would begin offering arm-based runtimes for github actions, but it seems like it's taking more time than I'd hoped 😕 |
Alternative workaroundUntil a solution is found to build an arm64 with github actions. For those like me who does not like to depend upon a community image, another workaround is to simply build the image locally. Either by cloning this repo, or copying the |
|
It doesn't look like there's been any updates on the GitHub actions for awhile, so I wanted to bump this. I was having the same issues as other people with Apple Silicon until I downloaded the files for the version I needed and built my own container image. |
|
Unfortunately it's still the case that getting the the arm images going in an automated fashion isn't possible with github actions. Running the ARM images for postgresql on intel platforms still doesn't work very well, and github actions only has runners for intel platform at the moment. When they roll out the arm runners, the major hurdles will be clear. For the time being, folks wanting to run on apple silicon will need to build them locally. We've been holding off on documenting this since it was not clear when the ARM runners would be made available, but it's been 6 or 8 months now that folks have been adopting apple silicon setups so it's probably time to get going on the documentation 😃 |
Would a self-hosted runner be an option in the meantime? Documentation is always a good thing to keep up to date. |
I'm not sure how it could be done securely in an automated fashion. The runner would need to be hosted in a trusted environment and ideally on a reliable host/service.
Absolutely! In this case, the documentation would be targeted specifically for folks that would like to build the images on Apple Silicon. And it would only be temporary until the environments for building native arm images becomes available. I guess it could be considered writing documentation for a temporary workaround. |
|
Actions allows you to build cross platform via https://github.com/docker/build-push-action You can use my example file for reference: https://github.com/netreconlab/parse-hipaa/blob/main/.github/workflows/docker-publish.yml In which I used to recently build: https://registry.hub.docker.com/repository/docker/netreconlab/parse-hipaa/tags?page=1&ordering=last_updated |
|
Thanks @cbaker6 but the problem is that the cross platform build uses qemu which currently doesn't function properly with postgres. I personally haven't tested it in a few months, but it may be time to test again to see if newer qemu updates might have solved the problem. |
|
@phillipross any specific issues with qemu? FWIW we have been building our own images as described by @cbaker6 for a few weeks and haven’t noticed anything wrong, but maybe we aren’t using a specific feature of postgis which manifests the problems… |
|
@sadams build errors like this from December 2020. I have not had time to try again since then.
|
|
@ImreSamu disabeling JIT worked on my test. |
|
As of October 2023, the status of Docker Postgis on Arm (as far as I know) is as follows:
Development repo:
Test images (amd64 + arm64) for the users: https://hub.docker.com/r/imresamu/postgis
Important note:The development repository currently mostly reflects my ideas, and I'm trying to incorporate the requirements that have accumulated over the past years. Because of this, everything is subject to change, and it's not guaranteed that everything will be exactly this way in the final version. Personal impressions:For now, I am satisfied with CircleCI, and I believe it can work well (at least it seems much better than QEMU). December 6, 2023 |
|
@ImreSamu you are wonderful, thanks! :) |
The postgis/postgis images currently don't have arm64 support. postgis/docker-postgis#216
|
@ImreSamu I'm still getting this issue on your image: imresamu/postgis-arm64:16-master It keep restarts. |
|
( ARM ) As a temporary ARM support solution: @lovung : It's possible that your database was initialized with a different PostgreSQL version or on a different architecture (like amd64) Because of these issues, everyone is advised to start testing with an empty PostgreSQL data directory and proceed step by step. |
|
Great work! Has there been any progress on this lately? :) |
|
We ran into a similar problem but colleagues reported that the 16-3.4 (amd64?) image seems to work better on MacOSX than the previous ones. Also there seemed to be differences where even the 15-3.4 worked on some Mac versions (M2) but on others (M3) but could, of course, have been other kinds of differences involved as well. |
|
Hmmm... No arm64 image yet? I have built one for another project before, but it's a pain. I'm trying to avoid that if possible... What what are the current challenges if any to make a arm64 version? Any gotchas or show stoppers? What are the issues holding this back? |
The default postgis image is unstable with M1/M2 arch because the build is for amd64 and it crashes the Qemu. For more details about the issue, see: postgis/docker-postgis#216
|
@James-von-Detroit Please take a minute to read the conversation, it has been explained and summarized more than once. @ImreSamu Can you give us a quick update how it's going? We can see burning activity in your testing repo, can we help with anything? |
|
Docker-PostGIS Status (March 24, 2023) It is important to note that I work on this in my spare time and there is no large company behind me sponsoring this. It is largely a learning process for me as well. Therefore, I apologize to those who expect quick solutions or do not understand why this process is slow. A relatively fresh and good news:
And why is this important? In the current process, every ARM Docker image costs 3 API calls ( check + push + manifest), In the meantime, I have made progress as time allowed:
Now, I just need to find 2-3 consecutive days to:
For those who would like to help now, I have written a suggestion on how you can assist with testing, here: That's it for now, and I hope the situation with the postgis arm images will be resolved soon. |
Why:
- For a long time, Postgis has only offered official linux/amd64 Docker
images, but now they're preparing to publish also linux/arm64 images.
postgis/docker-postgis#216 (comment)
- On M3 Macbook, the ARM version is much faster than the x86 version:
"lein kaocha" test run before:
Total duration: 74,10113 seconds
207 tests, 1768 assertions, 0 failures.
Top 3 slowest kaocha.type/clojure.test (74,09882 seconds, 100,0% of total time)
slow
2,92415 seconds average (55,55887 seconds / 19 tests)
e2e
17,50770 seconds average (17,50770 seconds / 1 tests)
fast
0,02064 seconds average (1,03225 seconds / 50 tests)
...and after:
Total duration: 47,63434 seconds
207 tests, 1768 assertions, 0 failures.
Top 3 slowest kaocha.type/clojure.test (47,63207 seconds, 100,0% of total time)
slow
1,46157 seconds average (27,76986 seconds / 19 tests)
e2e
18,94063 seconds average (18,94063 seconds / 1 tests)
fast
0,01843 seconds average (0,92158 seconds / 50 tests)
|
Any update on this? Would love to see the official PostGIS image support ARM. |
and others
I'm currently running Apple Silicon machine, which is using the ARM-64 ISA.
Currently the postgis image is only built for amd64.
Would it be possible to export ARM-64 builds?
The current workaround is to use community built images (e.g. https://github.com/Duvel/docker-postgis).
The text was updated successfully, but these errors were encountered: