Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Build more wheels for Linux, macOS and Windows #251

Merged
merged 4 commits into from
Feb 12, 2024
Merged

Build more wheels for Linux, macOS and Windows #251

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
0706000
Build more wheels for Linux, macOS and Windows
Schamper Feb 3, 2024
5702ce8
Process feedback
Schamper Feb 5, 2024
d4455a9
Add concurrency limit
Schamper Feb 5, 2024
d3b7a78
Add tags trigger
Schamper Feb 6, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
73 changes: 62 additions & 11 deletions .github/workflows/publish-to-pypi.yml
View file
Open in desktop
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You need to parametrize the name of the artifact, because you're adding a matrix, so several artifacts will be produced.

      - name: Store the distribution packages
        uses: actions/upload-artifact@v4
        with:
          name: python-package-distributions-${{ matrix.os }}-${{ matrix.arch }}
path: dist/*.whl

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And it's seems that the --output-dir dist/ attribute is loose 😅

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added the workflow, but I should have made sure that it could be run on a pull_request event.

Copy link
Contributor Author

@Schamper Schamper Feb 5, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't touch the triggers yet because I'm not sure what the preference of the team is, but I've proposed something that I personally like.

Thanks for the tip on the artifact name. I hadn't used it like that before, but I believe I also made the correct changes to the download-artifact step to have it work as expected.

I also believe dist is the default output directory, but I've re-added it for completeness.

Since the QEMU run is quite expensive, I've also added a concurrency limit that I used in my own CI for this.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you can add tags to the push event, so the job publish-to-pypi will work, otherwise LGTM

on:
  push:
    branches:
      - master
    tags:
  pull_request:
  workflow_dispatch:

Original file line number Diff line number Diff line change
@@ -1,31 +1,81 @@
name: Publish Python distribution to PyPI

on: push
on:
push:
branches:
- master
tags:
pull_request:
workflow_dispatch:

concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true

jobs:
build:
runs-on: ubuntu-22.04
name: Build wheels on ${{ matrix.os }} (${{ matrix.arch }})
runs-on: ${{ matrix.os }}
strategy:
matrix:
include:
- os: ubuntu-22.04
arch: x86_64
- os: ubuntu-22.04
arch: i686
- os: ubuntu-22.04
arch: aarch64
- os: macos-12
arch: x86_64 universal2
- os: windows-2022
arch: x86
before: vcpkg install openssl:x86-windows-static
env: LIB="C:\\vcpkg\\packages\\openssl_x86-windows-static\\lib" INCLUDE="C:\\vcpkg\\packages\\openssl_x86-windows-static\\include"
- os: windows-2022
arch: AMD64
before: vcpkg install openssl:x64-windows-static
env: LIB="C:\\vcpkg\\packages\\openssl_x64-windows-static\\lib" INCLUDE="C:\\vcpkg\\packages\\openssl_x64-windows-static\\include"

steps:
- uses: actions/checkout@v4
with:
submodules: true

- name: Set up Python
uses: actions/setup-python@v5
- name: Set up QEMU
if: runner.os == 'Linux'
uses: docker/setup-qemu-action@v2
with:
python-version: "3.x"

- name: Install cibuildwheel
run: python3 -m pip install cibuildwheel==2.16.4
platforms: all

- name: Build wheels
run: python -m cibuildwheel --output-dir dist/
uses: pypa/cibuildwheel@v2.16.5
with:
output-dir: dist
env:
CIBW_ARCHS: ${{ matrix.arch }}
CIBW_BEFORE_ALL_LINUX: |
if [[ ! -z "$(which yum)" ]]; then
yum install -y make gcc perl-core pcre-devel wget zlib-devel git automake
wget https://ftp.openssl.org/source/openssl-1.1.1k.tar.gz
tar xf openssl*.gz
cd openssl*
./config --prefix=/usr --openssldir=/etc/ssl zlib-dynamic
make -j$(nproc)
make install
elif [[ ! -z "$(which apk)" ]]; then
apk add openssl-dev
fi
CIBW_BEFORE_ALL_WINDOWS: ${{ matrix.before }}
CIBW_BUILD_FRONTEND: build
CIBW_CONFIG_SETTINGS: --enable-cuckoo --enable-magic --enable-dex --enable-macho --enable-openssl
CIBW_ENVIRONMENT: ${{ matrix.env }}
CIBW_SKIP: cp36-*
CIBW_TEST_COMMAND: python {package}/tests.py

- name: Store the distribution packages
uses: actions/upload-artifact@v4
with:
name: python-package-distributions
name: python-package-distributions-${{ matrix.os }}-${{ matrix.arch }}
path: dist/*.whl

publish-to-pypi:
Expand All @@ -45,7 +95,8 @@ jobs:
- name: Download all the dists
uses: actions/download-artifact@v4
with:
name: python-package-distributions
pattern: python-package-distributions-*
merge-multiple: true
path: dist/

- name: Publish distribution to PyPI
Expand Down
Loading