Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parallelize wheel-builds where applicable #314

Merged
merged 11 commits into from
Jul 20, 2022
47 changes: 27 additions & 20 deletions .github/workflows/wheel_linux_aarch64.yml
Original file line number Diff line number Diff line change
@@ -1,38 +1,27 @@
name: Wheel::Linux::ARM

# **What it does**: Builds python wheels for Linux (ubuntu-latest) architecture ARM 64 and store it as artifacts.
# Python versions: 3.7, 3.8, 3.9, 3.10.
# **Why we have it**: To build wheels for pennylane-lightning instalation.
AmintorDusko marked this conversation as resolved.
Show resolved Hide resolved
# **Who does it impact**: Wheels to be uploaded to PyPI.

on:
push:
branches:
- master
release:
types: [published]

env:
CIBW_BUILD: 'cp37-* cp38-* cp39-* cp310-*'
CIBW_SKIP: "*-musllinux*"

# Python build settings
CIBW_BEFORE_BUILD: |
pip install pybind11 ninja cmake && yum install -y gcc gcc-c++

# Testing of built wheels
CIBW_TEST_REQUIRES: numpy~=1.21 scipy pytest pytest-cov pytest-mock flaky

CIBW_BEFORE_TEST: pip install git+https://github.com/PennyLaneAI/pennylane.git@master

CIBW_TEST_COMMAND: |
pl-device-test --device=lightning.qubit --skip-ops -x --tb=short --no-flaky-report

# Use Centos 7 wheel-builder for ARM
CIBW_MANYLINUX_AARCH64_IMAGE: manylinux2014

jobs:
linux-wheels-aarch64:
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest]
arch: [aarch64]
name: ubuntu-latest::aarch64
cibw_build: [cp37-*, cp38-*, cp39-*, cp310-*]

name: ubuntu-latest::aarch64 (Python ${{ fromJson('{ "cp37-*":"3.7","cp38-*":"3.8","cp39-*":"3.9","cp310-*":"3.10" }')[matrix.cibw_build] }})
runs-on: ${{ matrix.os }}

steps:
Expand All @@ -57,6 +46,24 @@ jobs:
- name: Build wheels
run: python -m cibuildwheel --output-dir wheelhouse
env:
CIBW_BUILD: ${{ matrix.cibw_build }}
CIBW_SKIP: "*-musllinux*"

# Python build settings
CIBW_BEFORE_BUILD: |
pip install pybind11 ninja cmake && yum install -y gcc gcc-c++

# Testing of built wheels
CIBW_TEST_REQUIRES: numpy~=1.21 scipy pytest pytest-cov pytest-mock flaky

CIBW_BEFORE_TEST: pip install git+https://github.com/PennyLaneAI/pennylane.git@master

CIBW_TEST_COMMAND: |
pl-device-test --device=lightning.qubit --skip-ops -x --tb=short --no-flaky-report

# Use Centos 7 wheel-builder for ARM
CIBW_MANYLINUX_AARCH64_IMAGE: manylinux2014

CIBW_ARCHS_LINUX: ${{matrix.arch}}

- uses: actions/upload-artifact@v2
Expand Down
41 changes: 24 additions & 17 deletions .github/workflows/wheel_linux_ppc64le.yml
Original file line number Diff line number Diff line change
@@ -1,35 +1,27 @@
name: Wheel::Linux::PowerPC

# **What it does**: Build python wheels for Linux (ubuntu-latest) architecture PowerPC 64 and store it as artifacts.
# Python versions: 3.7, 3.8, 3.9, 3.10.
# **Why we have it**: To build wheels for pennylane-lightning instalation.
# **Who does it impact**: Wheels to be uploaded to PyPI.

on:
push:
branches:
- master
release:
types: [published]

env:
CIBW_BUILD: 'cp37-* cp38-* cp39-* cp310-*'
CIBW_SKIP: "*-musllinux*"

# Python build settings
CIBW_BEFORE_BUILD: |
pip install pybind11 ninja cmake && yum install -y gcc gcc-c++

# Skip PPC tests due to lack of numpy/scipy wheel support
CIBW_TEST_SKIP: "*-manylinux_{ppc64le}"

CIBW_BEFORE_TEST: pip install git+https://github.com/PennyLaneAI/pennylane.git@master

# Use CentOS 7 image for PPC
CIBW_MANYLINUX_PPC64LE_IMAGE: manylinux2014

jobs:
linux-wheels-ppc64le:
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest]
arch: [ppc64le]
name: ubuntu-latest::ppc64le
cibw_build: [cp37-*, cp38-*, cp39-*, cp310-*]

name: ubuntu-latest::ppc64le (Python ${{ fromJson('{ "cp37-*":"3.7","cp38-*":"3.8","cp39-*":"3.9","cp310-*":"3.10" }')[matrix.cibw_build] }})
runs-on: ${{ matrix.os }}

steps:
Expand All @@ -54,6 +46,21 @@ jobs:
- name: Build wheels
run: python -m cibuildwheel --output-dir wheelhouse
env:
CIBW_BUILD: ${{ matrix.cibw_build }}
CIBW_SKIP: "*-musllinux*"

# Python build settings
CIBW_BEFORE_BUILD: |
pip install pybind11 ninja cmake && yum install -y gcc gcc-c++

# Skip PPC tests due to lack of numpy/scipy wheel support
CIBW_TEST_SKIP: "*-manylinux_{ppc64le}"

CIBW_BEFORE_TEST: pip install git+https://github.com/PennyLaneAI/pennylane.git@master

# Use CentOS 7 image for PPC
CIBW_MANYLINUX_PPC64LE_IMAGE: manylinux2014

CIBW_ARCHS_LINUX: ${{matrix.arch}}

- uses: actions/upload-artifact@v2
Expand Down
55 changes: 31 additions & 24 deletions .github/workflows/wheel_linux_x86_64.yml
Original file line number Diff line number Diff line change
@@ -1,30 +1,17 @@
name: Wheel::Linux::x86_64

# **What it does**: Builds python wheels for Linux (ubuntu-latest) architecture x86_64 and store it as artifacts.
# Python versions: 3.7, 3.8, 3.9, 3.10.
# **Why we have it**: To build wheels for pennylane-lightning instalation.
# **Who does it impact**: Wheels to be uploaded to PyPI.

on:
release:
types: [published]
pull_request:
push:
branches:
- master
pull_request:

env:
CIBW_BUILD: 'cp37-* cp38-* cp39-* cp310-*'
CIBW_SKIP: "*-musllinux*"

# Python build settings
CIBW_BEFORE_BUILD: |
pip install pybind11 ninja cmake && yum install -y gcc gcc-c++

# Testing of built wheels
CIBW_TEST_REQUIRES: numpy~=1.21 scipy pytest pytest-cov pytest-mock flaky

CIBW_BEFORE_TEST: |
pip install git+https://github.com/PennyLaneAI/pennylane.git@master

CIBW_TEST_COMMAND: |
pl-device-test --device=lightning.qubit --skip-ops -x --tb=short --no-flaky-report

CIBW_MANYLINUX_X86_64_IMAGE: manylinux2014
release:
types: [published]

jobs:
linux-wheels-x86-64:
Expand All @@ -33,7 +20,9 @@ jobs:
matrix:
os: [ubuntu-latest]
arch: [x86_64]
name: ${{ matrix.os }}
cibw_build: [cp37-*, cp38-*, cp39-*, cp310-*]

name: ${{ matrix.os }} (Python ${{ fromJson('{ "cp37-*":"3.7","cp38-*":"3.8","cp39-*":"3.9","cp310-*":"3.10" }')[matrix.cibw_build] }})
runs-on: ${{ matrix.os }}

steps:
Expand All @@ -53,10 +42,28 @@ jobs:
run: python -m pip install cibuildwheel==2.3.0

- name: Build wheels
run: python -m cibuildwheel --output-dir wheelhouse
env:
CIBW_BUILD: ${{ matrix.cibw_build }}
CIBW_SKIP: "*-musllinux*"

# Python build settings
CIBW_BEFORE_BUILD: |
pip install pybind11 ninja cmake && yum install -y gcc gcc-c++

# Testing of built wheels
CIBW_TEST_REQUIRES: numpy~=1.21 scipy pytest pytest-cov pytest-mock flaky

CIBW_BEFORE_TEST: |
pip install git+https://github.com/PennyLaneAI/pennylane.git@master

CIBW_TEST_COMMAND: |
pl-device-test --device=lightning.qubit --skip-ops -x --tb=short --no-flaky-report

CIBW_MANYLINUX_X86_64_IMAGE: manylinux2014
CIBW_ARCHS_LINUX: ${{matrix.arch}}

run: python -m cibuildwheel --output-dir wheelhouse

- uses: actions-ecosystem/action-regex-match@v2
id: rc_build
with:
Expand Down
57 changes: 33 additions & 24 deletions .github/workflows/wheel_macos_arm64.yml
Original file line number Diff line number Diff line change
@@ -1,43 +1,31 @@
name: Wheel::MacOS::ARM

# **What it does**: Builds python wheels for MacOS (11) architecture ARM 64 and store it as artifacts.
# Python versions: 3.8, 3.9, 3.10.
# **Why we have it**: To build wheels for pennylane-lightning instalation.
# **Who does it impact**: Wheels to be uploaded to PyPI.

on:
pull_request:
push:
branches:
- master
pull_request:
release:
types: [published]

env:
CIBW_BUILD: 'cp37-* cp38-* cp39-* cp310-*'

ARCHS: 'arm64'

# MacOS specific build settings
CIBW_BEFORE_ALL_MACOS: |
brew uninstall --force oclint

# Python build settings
CIBW_BEFORE_BUILD: |
pip install pybind11 ninja cmake

# Testing of built wheels
CIBW_TEST_REQUIRES: numpy~=1.21 scipy pytest pytest-cov pytest-mock flaky

CIBW_BEFORE_TEST: pip install git+https://github.com/PennyLaneAI/pennylane.git@master

CIBW_TEST_COMMAND: |
pl-device-test --device=lightning.qubit --skip-ops -x --tb=short --no-flaky-report

CIBW_BUILD_VERBOSITY: 1

jobs:
mac-wheels-arm64:
strategy:
fail-fast: false
matrix:
os: [macos-11]
arch: [arm64]
name: macos-latest::arm64
cibw_build: [cp38-*, cp39-*, cp310-*]

name: macos-latest::arm64 (Python ${{ fromJson('{ "cp37-*":"3.7","cp38-*":"3.8","cp39-*":"3.9","cp310-*":"3.10" }')[matrix.cibw_build] }})
mlxd marked this conversation as resolved.
Show resolved Hide resolved
runs-on: ${{ matrix.os }}

steps:
Expand All @@ -57,11 +45,32 @@ jobs:
run: python -m pip install cibuildwheel==2.3.0

- name: Build wheels
run: |
python -m cibuildwheel --output-dir wheelhouse
env:
CIBW_BUILD: ${{ matrix.cibw_build }}

# MacOS specific build settings
CIBW_BEFORE_ALL_MACOS: |
brew uninstall --force oclint

# Python build settings
CIBW_BEFORE_BUILD: |
pip install pybind11 ninja cmake

# Testing of built wheels
CIBW_TEST_REQUIRES: numpy~=1.21 scipy pytest pytest-cov pytest-mock flaky

CIBW_BEFORE_TEST: pip install git+https://github.com/PennyLaneAI/pennylane.git@master

CIBW_TEST_COMMAND: |
pl-device-test --device=lightning.qubit --skip-ops -x --tb=short --no-flaky-report

CIBW_BUILD_VERBOSITY: 1

CIBW_ARCHS_MACOS: ${{ matrix.arch }}

run: |
python -m cibuildwheel --output-dir wheelhouse

- uses: actions/upload-artifact@v2
if: ${{ github.event_name == 'release' || github.ref == 'refs/heads/master' }}
with:
Expand Down
58 changes: 34 additions & 24 deletions .github/workflows/wheel_macos_x86_64.yml
Original file line number Diff line number Diff line change
@@ -1,34 +1,19 @@
name: Wheel::MacOS::Intel

# **What it does**: Builds python wheels for MacOS (10.15) architecture x86_64 and store it as artifacts.
# Python versions: 3.7, 3.8, 3.9, 3.10.
# **Why we have it**: To build wheels for pennylane-lightning instalation.
# **Who does it impact**: Wheels to be uploaded to PyPI.

on:
pull_request:
push:
branches:
- master
pull_request:
release:
types: [published]

env:
CIBW_BUILD: 'cp37-* cp38-* cp39-* cp310-*'

# MacOS specific build settings
CIBW_BEFORE_ALL_MACOS: |
brew uninstall --force oclint
brew install libomp

# Python build settings
CIBW_BEFORE_BUILD: |
pip install pybind11 ninja cmake

# Testing of built wheels
CIBW_TEST_REQUIRES: numpy~=1.21 scipy pytest pytest-cov pytest-mock flaky

CIBW_BEFORE_TEST: pip install git+https://github.com/PennyLaneAI/pennylane.git@master

CIBW_TEST_COMMAND: |
pl-device-test --device=lightning.qubit --skip-ops -x --tb=short --no-flaky-report

CIBW_BUILD_VERBOSITY: 1

MACOSX_DEPLOYMENT_TARGET: 10.15

jobs:
Expand All @@ -38,7 +23,9 @@ jobs:
matrix:
os: [macos-10.15]
arch: [x86_64]
name: ${{ matrix.os }}
cibw_build: [cp37-*, cp38-*, cp39-*, cp310-*]

name: ${{ matrix.os }} (Python ${{ fromJson('{ "cp37-*":"3.7","cp38-*":"3.8","cp39-*":"3.9","cp310-*":"3.10" }')[matrix.cibw_build] }})
runs-on: ${{ matrix.os }}

steps:
Expand All @@ -58,11 +45,34 @@ jobs:
run: python -m pip install cibuildwheel==2.3.0

- name: Build wheels
run: python -m cibuildwheel --output-dir wheelhouse
env:
CIBW_BUILD: ${{ matrix.cibw_build }}

# MacOS specific build settings
CIBW_BEFORE_ALL_MACOS: |
brew uninstall --force oclint
brew install libomp

# Python build settings
CIBW_BEFORE_BUILD: |
pip install pybind11 ninja cmake

# Testing of built wheels
CIBW_TEST_REQUIRES: numpy~=1.21 scipy pytest pytest-cov pytest-mock flaky

CIBW_BEFORE_TEST: pip install git+https://github.com/PennyLaneAI/pennylane.git@master

CIBW_TEST_COMMAND: |
pl-device-test --device=lightning.qubit --skip-ops -x --tb=short --no-flaky-report

CIBW_BUILD_VERBOSITY: 1

CIBW_ARCHS_MACOS: ${{matrix.arch}}

USE_OMP: 1

run: python -m cibuildwheel --output-dir wheelhouse

- uses: actions-ecosystem/action-regex-match@v2
id: rc_build
with:
Expand Down
Loading