Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use a matrix for library build in release workflow #1535

Closed
wants to merge 2 commits into from

Conversation

connec
Copy link

@connec connec commented Feb 26, 2022

I'm working on a Go application, using an M1 mac and an Alpine Linux container-based local development environment. I'd like to use Oso, but to do so I need linux arm64 support in the Go package. I've successfully spiked this locally, and started investigating how to bring it upstream.

In the process, I decided to take a stab at simplifying the current build by using a matrix of all the builds, and their shared/static library artifacts. This is intended to make it easier to see what targets are currently being built, and to simplify adding others in future.

It additionally introduces more parallelism, since each target will build in parallel vs. each OS before (and atm each OS has two builds). On the down side, it's not currently possible to depend on specific matrix job instances, so some jobs have to wait for additional builds. In theory, the parallelism should cancel this out, for a net win.

It's possible the naming conventions could be standardised such that the shared/static fields could just be boolean, but this way no changes are needed in the other jobs. Something to consider in future.

I've been testing it in my fork but haven't achieved a fully green run yet. I'm not set up to for the m1 mac runner (are there any instructions for this?), and I'm seeing missing Python 3.6 on Windows, plus a number of seemingly spurious errors. Not sure if any of these are known issues?

image

I see there's some activity around this already (e.g. #1529 and #808 (comment)), so hopefully this doesn't interfere with any WIP.

@github-actions
Copy link

github-actions bot commented Feb 26, 2022

CLA Assistant Lite bot All contributors have signed the CLA ✍️ ✅

@connec
Copy link
Author

connec commented Feb 26, 2022

I have read the CLA Document and I hereby sign the CLA

@connec connec force-pushed the build-matrix branch 2 times, most recently from 8e63715 to d0a7b4f Compare March 8, 2022 14:41
This is intended to make it easier to see what targets are currently
being built, and to simplify adding others in future.

It additionally introduces more parallelism, since each target will
build in parallel vs. each OS before (and atm each OS has two builds).
On the down side, it's not currently possible to depend on specific
matrix job instances, so some jobs have to wait for additional builds.
In theory, the parallelism should cancel this out, for a net win.

It's possible the naming conventions could be standardised such that the
`shared`/`static` fields could just be `boolean`, but this way no
changes are needed in the other jobs. Something to consider in future.
This is apparently no longer available, per the error from the
`actions/setup-python@v1` step:

    Error: Version 3.6 with arch x64 not found
    Available versions:
    3.10.2 (x86)
    3.7.9 (x86)
    3.8.10 (x86)
    3.9.10 (x86)
    3.10.2 (x64)
    3.7.9 (x64)
    3.8.10 (x64)
    3.9.10 (x64)
@connec
Copy link
Author

connec commented Mar 8, 2022

The release now goes through if I host an m1 runner: https://github.com/connec/oso/actions/runs/1952256611

image

(dispatch still fails because it depends on secrets that aren't on my fork, and release was skipped because it's on main not a tag)

@connec
Copy link
Author

connec commented Mar 8, 2022

Note though that I've removed the validation for Python 3.6 on Windows, since this doesn't seem to be supported by actions/setup-python@v1:

Error: Version 3.6 with arch x64 not found
Available versions:
3.10.2 (x86)
3.7.9 (x86)
3.8.10 (x86)
3.9.10 (x86)
3.10.2 (x64)
3.7.9 (x64)
3.8.10 (x64)
3.9.10 (x64)

@connec
Copy link
Author

connec commented Apr 18, 2022

Closing per #1540.

@connec connec closed this Apr 18, 2022
@github-actions github-actions bot locked and limited conversation to collaborators Apr 18, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant