Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

arm64 multiarch BuildKit compilation randomly segfaults #1309

Closed
3 tasks done
FacuM opened this issue Jan 26, 2025 · 5 comments
Closed
3 tasks done

arm64 multiarch BuildKit compilation randomly segfaults #1309

FacuM opened this issue Jan 26, 2025 · 5 comments

Comments

@FacuM
Copy link

FacuM commented Jan 26, 2025

Contributing guidelines

I've found a bug, and:

  • The documentation does not mention anything about my problem
  • There are no open or closed issues that are related to my problem

Description

Compilling my multiarch project started to randomly segfault on the action's QEMU binary while processing the Dockerfile steps.

As this is a public project, you can actually just take a look at the job output, but may also check the build log attached below. If you were to retry that job, it would fail on another random spot.

I don't know for how long this has been a thing since I haven't touched the image for about a month.

Expected behaviour

Getting the resulting image and publishing it to Docker Hub.

Actual behaviour

Random crashes and segfaults during image compilation for the target system, amd64 looks to be alright, but arm64 (which runs on top of QEMU) fails. I've already tried upgrading my dependencies, including both docker/setup-buildx-action and docker/build-push-action.

Repository URL

https://github.com/wprint3d/wprint3d

Workflow run URL

https://github.com/wprint3d/wprint3d/blob/alpha/.github/workflows/docker-image.yml

YAML workflow

name: Docker Image CI

on:
  push:
    branches: [ "alpha" ]
  pull_request:
    branches: [ "alpha" ]

concurrency:
  group: ${{ github.ref }}
  cancel-in-progress: false

jobs:

  build:
    runs-on: ubuntu-latest
    steps:
      -
        name: Checkout
        uses: actions/checkout@v3
      -
        name: Login to Docker Hub
        uses: docker/login-action@v2
        with:
          username: ${{ secrets.DOCKERHUB_USERNAME }}
          password: ${{ secrets.DOCKERHUB_TOKEN }}
      -
        name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3
      -
        name: Build and push
        uses: docker/build-push-action@v6
        with:
          context: .
          file: ./Dockerfile
          platforms: linux/amd64,linux/arm64
          push: true
          tags: ${{ secrets.DOCKERHUB_USERNAME }}/wprint3d:latest
          cache-from: type=gha
          cache-to: type=gha,mode=max

Workflow logs

logs_33538460101.zip

BuildKit logs

BuildKit logs are contained as part of the attached ZIP file and the link to the job.

Additional info

No response

@kahara
Copy link

kahara commented Jan 27, 2025

Observing the same thing here, I think:

https://github.com/kahara/vushf-exporter/actions/runs/12996988416/job/36247696081#step:10:401

On 2025-01-19, was able to run jobs to completion several times without issues. So perhaps things worked fine at least five seven days ago:

https://github.com/kahara/vushf-exporter/actions

Today there was first a single successful run but after that, each one has failed.

@FacuM
Copy link
Author

FacuM commented Jan 27, 2025

Observing the same thing here, I think:

https://github.com/kahara/vushf-exporter/actions/runs/12996988416/job/36247696081#step:10:401

On 2025-01-19, was able to run jobs to completion several times without issues. So perhaps things worked fine at least five seven days ago:

https://github.com/kahara/vushf-exporter/actions

Today there was first a single successful run but after that, each one has failed.

Okay, I have a temporary workaround! Try using ubuntu-22.04 instead of ubuntu-latest or ubuntu-24.04.

jobs:

  build:
    runs-on: ubuntu-22.04

Please try like that and let me know if it helps. :)

@crazy-max
Copy link
Member

Not build-push-action but QEMU related similar to docker/setup-qemu-action#188

This is tracked in tonistiigi/binfmt#215

You can try using qemu v8 to see if this fixes the issue for you: tonistiigi/binfmt#215 (comment)

- name: Set up QEMU
  uses: docker/setup-qemu-action@v3
  with:
    image: tonistiigi/binfmt:qemu-v8.1.5

Or switch to ubuntu-22.04 runner.

kahara added a commit to kahara/vushf-exporter that referenced this issue Jan 28, 2025
kahara added a commit to kahara/vushf-exporter that referenced this issue Jan 28, 2025
* cache image for docker/setup-qemu-action

* use v8 tonistiigi/binfmt

As seen in:

docker/build-push-action#1309 (comment)

tonistiigi/binfmt#215 (comment)
@kahara
Copy link

kahara commented Jan 28, 2025

Went with the "v8" approach, and that did it. Thanks everyone!

@FacuM
Copy link
Author

FacuM commented Feb 2, 2025

I do understand that this is caused by a broken external dependency, but why is it closed? This IS still an issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants