From 01187c148b494a54cfff1779b121b59a10ed2e0a Mon Sep 17 00:00:00 2001 From: Vincent Michaud-Rioux Date: Tue, 7 May 2024 13:45:55 -0400 Subject: [PATCH] Accelerate CI parallelizing over OMP configs (#712) * Update Changelog * update release candidate version * run agains pennylane rc * update relese cadidate version * Increase tolerance for a flaky test (test_single_return_value) * Update `LightningBaseFallBack` to raise an error instead of falling back to default.qubit (#689) * Initial commit. * Update changelog * Auto update version * trigger ci * Unused import. * Auto update version * trigger ci * Fix imports. * Auto update version * Fix LightningException * make format * Auto update version from '0.36.0-dev34' to '0.36.0-dev35' * Auto update version from '0.36.0-dev35' to '0.36.0-dev36' * Auto update version from '0.36.0-dev36' to '0.36.0-dev37' * Make python -m pip install consistent. * Fix ImportError * Auto update version from '0.36.0-dev37' to '0.36.0-dev38' * --force-reinstall * Auto update version from '0.36.0-dev38' to '0.36.0-dev39' * Remove --force-reinstall * Auto update version from '0.36.0-dev39' to '0.36.0-dev40' * Auto update version from '0.36.0-dev42' to '0.36.0-dev43' * Auto update version from '0.36.0-dev43' to '0.36.0-dev44' * Spec CMAKE_CXX_COMPILER * Fix quotes. * Remove quotes * Hardcode g++-11 * use env.COMPILER * which g++ * Revert * Downgrade pip * Pin libomp. * Revert wheel_linux * Upload artifacts. * Comment upload trigger * Install pip==23 * Revert libomp * try testing outside cibuildwheel * yum install devtools * yum install devtools * Do not install python * Revert changes * Don't editable install. * Not edit install * Not edit install * Fix cov path * Explicit mcm tests path * Fix qml.QuantumFunctionError error messages. --------- Co-authored-by: Dev version update bot Co-authored-by: ringo-but-quantum Co-authored-by: AmintorDusko * trigger CIs * Introduce pytest-splits (#696) * Try update lightning CI paths * Auto update version * Update kokkos tests to trigger on changes to pl core * Auto update version * Split tests_linux.yml into cpp/python. * Auto update version from '0.36.0-dev36' to '0.36.0-dev38' * Split GPU tests into cpp/python. * Split MPI-GPU tests into cpp/python. * Fix LGPU triggers. * Remove python tests from LGPU-C++. * Remove upload-to-codecov-linux-cpp from LGPUMPI tests. * Rename test workflows. * Update changelog. * Change group names. * Use paths-ignore to simplify workflow paths * Auto update version from '0.36.0-dev38' to '0.36.0-dev39' * Fix test names. * you may only define one of `paths` and `paths-ignore` for a single event * Fix formatting * Auto update version from '0.36.0-dev39' to '0.36.0-dev40' * Change to paths-ignore in no-bin tests. * Auto update version from '0.36.0-dev40' to '0.36.0-dev41' * trigger ci * Change error message. * Split build/test jobs. * Revert pennylane_lightning/lightning_kokkos/lightning_kokkos.py * Fix setup.py command * Comment workflows * Comment pull_request: * Copy wheels to github.workspace /wheels * SKIP_COMPILATION=True * Revert SKIP_COMPILATION=True * Remove duplicate python actions. * Split LQ-test into 5 groups. * Update requirements-dev.txt * 2 groups * 1 group * 7 groups * Refactor LKokos tests * Auto update version from '0.36.0-dev42' to '0.36.0-dev43' * Fix cache id * Fix needs * Remove all builds * put back CMAKE_PREFIX_PATH * install fake lightning * Fix compat paths * Auto update version from '0.36.0-dev42' to '0.36.0-dev43' * Revert changes. * Update changelog * trigger ci * build_and_cache_Kokkos_linux.yml not concurrent. * Remove C++ tests from compat tests (skipped anyways). * Use ${{ github.ref }}-${{ github.event }}-${{ inputs.lightning-version }}-${{ inputs.pennylane-version }} for concurrency groups. * PL_BACKEND: ${{ matrix.pl_backend }} * Update changelog * Install wheel package. * Auto update version from '0.36.0-dev43' to '0.36.0-dev44' * Install scipy wheel in one place. * Comment job triggers. * Add LK[cpu] est durations. * trigger ci * Use sha independent keys. * trigger ci * build_and_cache_Kokkos * Change LK keys * ls -l * Install version nonce * Do not skip comp. * Trade ref for sha in key * Install after PL * ref ==> sha * Reorder * Try LQ workflows. * Use actions/setup-python * Revert triggers. * Provide correct cov path. * Revert pennylane_lightning/core/src/utils/config.h * LK uses 5 groups. * Restore wheels to main * main/wheels * Revert cache paths * Fix ls -l * fail-on-cache-miss: true * Introduce env.GITHUB_WORKSPACE * Use artifacts * Fix paths * No ls -l * Download to github.workspace * fix wheel name * WHEEL_NAME * Add blas. * Use outputs. * Send wheel name * Fix artifact name * Fix name * Use WHEEL_NAME var * artifacts kokkos * fix wheel name * Restore bins. * Clean up * Split Linux CPU python tests into 2. * Fix compat workflows. * Split format and tidy workflows. * fix lkcpu * fix concurrency group * Do not use editable installs. * Split full kokkos tests except unitary_correct * Upload LK test durations. * duration-paths * Unique test_durations * Unique test_durations * Unique test_durations * Fix path * Update python_lightning_kokkos_test_durations * Remove obsolete all and SERIAL tests. * Update changelog * Creating the `LightningTensor` device class based on the new API (#671) * empty commit (triggering CI) * Auto update version * Definition of the two front-end classes * adding the `lightning_tensor` string to the supported backends in `setup.py` * adding `__init__.py` file to directly import the `lightning_tensor` device class * re-naming file * Auto update version * Creating the first prototype of initial MPS state tensor using `quimb` * providing the `backend`, `method` parameters and making `wires` optional * Changing names and structure * Auto update version * adding method required by the new device API design * Auto update version * using the `kwargs` parameter in `LightningTensor` and `CircuitMPS` in `quimb` * taking some further inputs from the new device API * Perhaps decided the overall structure of `LIghtningTensor` * Auto update version * adding docs to methods * temporary changes so that `pylint` does not complain at this stage * running `isort` * re-running formatter after `isort` * re-running formatter after `isort` * Applying suggested formatting change from CI * adding tmp unit tests * Adding `quimb` in `requirements.txt` * runing `isort` on mps test * removing `quimb` from requirement and deleting unit tests for `lightning.tensor` * Auto update version * re-inserting unit tests with an additional `yml` file * running isort on quimb test * changing name of yml file * preventing error in import * updating yml file * inserting `quimb` package in requirements-dev * strange error with `quimb` * strange error with `quimb` * specifying scipy version * removing installation of scipy from yml file * removing the new `yml` file * testing if tests are tested * Covering all lines in tests * forgot final line for formatter * Python formatter on CI complaints * covering missing lines * formatter on CI complaints * Trying not to skip test if Cpp is enabled * skipping tests if Cpp is enabled * removing the only line not covered by tests so far * Auto update version * Applying suggestions from code review and making the `state` attribute private (new API design) * Python formatter * removing params from `QuimbMPS` * Auto update version * removing `**kwargs` from `QuimbMPS` * removing unnecessary param at this stage * covering test line * formatter... * removing param description * Making `pylint` happy * forgot new arg in test * Updating base class and `preprocess` function * Updating `LightningTensor` class with new names from more advanced PR * Auto update version * Auto update version * Triggering CI * Auto update version * Trying to remove pin from `quimb` in `requirements.dev` * Auto update version * Auto update version * Removing infos on derivatives and using config options to pass parameters to interface * Usual `pylint` failures * Trying to solve formatting errors * typo in docstring * Sunday update: improved docstrings and structure * Removing method that was supposed to be in next PR * removing old TODO comment * Removing changes from the `setup.py` file * restoring previous format to `setup.py` * Auto update version from '0.36.0-dev34' to '0.36.0-dev41' * Auto update version from '0.36.0-dev40' to '0.36.0-dev41' * Removing kwargs as suggested from code review * Addressing comments from CR * Skipping tests if CPP binary is available * Auto update version from '0.36.0-dev42' to '0.36.0-dev43' * Auto update version from '0.36.0-dev43' to '0.36.0-dev44' * Restoring name in changelog (?) * Increasing time limit for Python tests * Applying suggestions from code review --------- Co-authored-by: Dev version update bot Co-authored-by: ringo-but-quantum * Increase tolerance for a flaky test (test_single_return_value) (#703) * update dev version * increase tolerance * update changelog * trigger CIs * Auto update version from '0.36.0-dev44' to '0.36.0-dev45' --------- Co-authored-by: ringo-but-quantum * Auto update version from '0.36.0-dev45' to '0.36.0-dev46' * Remove LTensor stuff * Update changelog * Revert requirements-dev.txt * trigger ci * Update python_lightning_kokkos_test_durations * Restore 'all' tests in tests_lkcuda_python.yml * Fix durations paths. * Try --splitting-algorithm=least_duration * Try no -n auto lkcpu tests * One last test. --------- Co-authored-by: Rashid N H M <95639609+rashidnhm@users.noreply.github.com> Co-authored-by: Dev version update bot Co-authored-by: ringo-but-quantum Co-authored-by: Pietropaolo Frisoni Co-authored-by: Amintor Dusko <87949283+AmintorDusko@users.noreply.github.com> * Update changelog (#706) * Use wire order specified on instantiation (#705) * use map wires instead of map_to_standard_wires * oops * Update .github/CHANGELOG.md * black * Trigger CIs * add test * black on tests * Update .github/CHANGELOG.md --------- Co-authored-by: AmintorDusko Co-authored-by: Ali Asadi <10773383+maliasadi@users.noreply.github.com> * Update wire order support for probabilities (#707) * Begin expansion of probs tests * Update tests to exhaustively compute probs with wires * Update all tests in LQ Meas to use CHECK instead of REQUIRES for better signal of failures * Add Jet Permuter * Add noisy functionality * Fix permutation ordering * Update changelog * Fix wrong ordering in probs tests * Add recommendation from apache flink to purge azure dirs * Forbid out of order indices for probs with LK and LG in CPP layer * Remove MSFT tools from GH Actions * Just try to remove dotnet * List opt hostedtoolcache * Remove packages from /opt/hostedtoolcache * Fix missing test hit * Forbid and remove broken tests from LK probs ordering * Fix clang-tidy isms * Ensure libopenblas is not broken from failed apt pull with dev * Appease GH Action * Full remove GH action modifier step due to unexplained failures * Add bidreictional wire validation for LGPU CPP layer * Clang-tidy warning fix * Appease CT * Update JET license agreement * Sync version * pacify CodeFactor * make CodeFactor ignore all JET code * trigger ci * trigger ci * Split OMP builds using matrix. * trigger ci * Ignore .github but not self. * Fix cd.. * Fix path * Update changelog [skip ci] * Revert changes to reqs.txt * Auto update version from '0.37.0-dev0' to '0.37.0-dev2' * Auto update version from '0.37.0-dev1' to '0.37.0-dev2' * Update .github/CHANGELOG.md Co-authored-by: Amintor Dusko <87949283+AmintorDusko@users.noreply.github.com> * Update .github/CHANGELOG.md Co-authored-by: Ali Asadi <10773383+maliasadi@users.noreply.github.com> --------- Co-authored-by: AmintorDusko Co-authored-by: Dev version update bot Co-authored-by: ringo-but-quantum Co-authored-by: Rashid N H M <95639609+rashidnhm@users.noreply.github.com> Co-authored-by: Pietropaolo Frisoni Co-authored-by: Amintor Dusko <87949283+AmintorDusko@users.noreply.github.com> Co-authored-by: Ali Asadi <10773383+maliasadi@users.noreply.github.com> Co-authored-by: Christina Lee Co-authored-by: Lee James O'Riordan --- .github/CHANGELOG.md | 17 +- .github/workflows/tests_lgpu_python.yml | 2 + .github/workflows/tests_lgpumpi_python.yml | 2 + .github/workflows/tests_linux_cpp.yml | 69 ++-- .github/workflows/tests_linux_python.yml | 440 --------------------- .github/workflows/tests_lkcpu_python.yml | 20 +- .github/workflows/tests_lkcuda_python.yml | 2 + .github/workflows/tests_lqcpu_python.yml | 21 +- .github/workflows/tidy.yml | 3 +- 9 files changed, 72 insertions(+), 504 deletions(-) delete mode 100644 .github/workflows/tests_linux_python.yml diff --git a/.github/CHANGELOG.md b/.github/CHANGELOG.md index 3e6a6dca5b..08805c5d4e 100644 --- a/.github/CHANGELOG.md +++ b/.github/CHANGELOG.md @@ -32,6 +32,9 @@ Amintor Dusko, Shuli Shu ### Improvements +* The various OpenMP configurations of Lightning-Qubit are tested in parallel on different Github Actions runners. + [(#712)](https://github.com/PennyLaneAI/pennylane-lightning/pull/712) + * Update Linux wheels to use manylinux_2_28 images. [(#667)](https://github.com/PennyLaneAI/pennylane-lightning/pull/667) @@ -43,7 +46,7 @@ Amintor Dusko, Shuli Shu This release contains contributions from (in alphabetical order): -Amintor Dusko +Amintor Dusko, Vincent Michaud-Rioux --- @@ -95,7 +98,6 @@ Amintor Dusko * Split Lightning-Qubit and Lightning-Kokkos CPU Python tests with `pytest-split`. Remove `SERIAL` from Kokkos' `exec_model` matrix. Remove `all` from Lightning-Kokkos' `pl_backend` matrix. Move `clang-tidy` checks to `tidy.yml`. Avoid editable `pip` installations. [(#696)](https://github.com/PennyLaneAI/pennylane-lightning/pull/696) - * Update `lightning.gpu` and `lightning.kokkos` to raise an error instead of falling back to `default.qubit`. [(#689)](https://github.com/PennyLaneAI/pennylane-lightning/pull/689) @@ -193,11 +195,20 @@ Amintor Dusko * Update the version of `codecov-action` to v4 and fix the CodeCov issue with the PL-Lightning check-compatibility actions. [(#682)](https://github.com/PennyLaneAI/pennylane-lightning/pull/682) +* Refactor of dev prerelease auto-update-version workflow. + [(#685)](https://github.com/PennyLaneAI/pennylane-lightning/pull/685) + +* Remove gates unsupported by catalyst from toml file. + [(#698)](https://github.com/PennyLaneAI/pennylane-lightning/pull/698) + +* Increase tolerance for a flaky test. + [(#703)](https://github.com/PennyLaneAI/pennylane-lightning/pull/703) + ### Contributors This release contains contributions from (in alphabetical order): -Ali Asadi, Amintor Dusko, Christina Lee, Vincent Michaud-Rioux, Lee James O'Riordan, Mudit Pandey, Shuli Shu +Ali Asadi, Amintor Dusko, Thomas Germain, Christina Lee, Erick Ochoa Lopez, Vincent Michaud-Rioux, Rashid N H M, Lee James O'Riordan, Mudit Pandey, Shuli Shu --- diff --git a/.github/workflows/tests_lgpu_python.yml b/.github/workflows/tests_lgpu_python.yml index e3c34a3188..ab49c7b141 100644 --- a/.github/workflows/tests_lgpu_python.yml +++ b/.github/workflows/tests_lgpu_python.yml @@ -13,6 +13,8 @@ on: release: pull_request: paths-ignore: + - .github + - '!.github/workflows/tests_lgpu_python.yml' - pennylane_lightning/core/src/simulators/lightning_kokkos/** - pennylane_lightning/core/src/simulators/lightning_qubit/** - pennylane_lightning/lightning_kokkos/** diff --git a/.github/workflows/tests_lgpumpi_python.yml b/.github/workflows/tests_lgpumpi_python.yml index 0830727e66..7f9a5e431f 100644 --- a/.github/workflows/tests_lgpumpi_python.yml +++ b/.github/workflows/tests_lgpumpi_python.yml @@ -16,6 +16,8 @@ on: - main pull_request: paths-ignore: + - .github + - '!.github/workflows/tests_lgpumpi_python.yml' - pennylane_lightning/core/src/simulators/lightning_kokkos/** - pennylane_lightning/core/src/simulators/lightning_qubit/** - pennylane_lightning/lightning_kokkos/** diff --git a/.github/workflows/tests_linux_cpp.yml b/.github/workflows/tests_linux_cpp.yml index 9247cedbcb..b66d8edec1 100644 --- a/.github/workflows/tests_linux_cpp.yml +++ b/.github/workflows/tests_linux_cpp.yml @@ -45,6 +45,11 @@ jobs: strategy: matrix: pl_backend: ["lightning_qubit"] + enable_kernel_omp: ["OFF", "ON"] + enable_kernel_avx_stream: ["OFF", "ON"] + exclude: + - enable_kernel_omp: OFF + enable_kernel_avx_stream: ON timeout-minutes: 60 name: C++ tests runs-on: ${{ needs.determine_runner.outputs.runner_group }} @@ -66,65 +71,39 @@ jobs: - name: Build and run unit tests run: | cmake . -BBuild -G Ninja \ - -DCMAKE_BUILD_TYPE=Debug \ - -DBUILD_TESTS=ON \ - -DENABLE_PYTHON=OFF \ - -DPL_BACKEND=${{ matrix.pl_backend }} \ - -DCMAKE_CXX_COMPILER=$(which g++-$GCC_VERSION) \ - -DENABLE_COVERAGE=ON - - cmake . -BBuildKernelOMP -G Ninja \ - -DCMAKE_BUILD_TYPE=Debug \ - -DBUILD_TESTS=ON \ - -DENABLE_PYTHON=OFF \ - -DPL_BACKEND=${{ matrix.pl_backend }} \ - -DCMAKE_CXX_COMPILER=$(which g++-$GCC_VERSION) \ - -DENABLE_COVERAGE=ON \ - -DLQ_ENABLE_KERNEL_OMP=ON - - cmake . -BBuildKernelAVXStream -G Ninja \ -DCMAKE_BUILD_TYPE=Debug \ -DBUILD_TESTS=ON \ -DENABLE_PYTHON=OFF \ -DPL_BACKEND=${{ matrix.pl_backend }} \ -DCMAKE_CXX_COMPILER=$(which g++-$GCC_VERSION) \ -DENABLE_COVERAGE=ON \ - -DLQ_ENABLE_KERNEL_AVX_STREAM=ON \ - -DLQ_ENABLE_KERNEL_OMP=ON + -DLQ_ENABLE_KERNEL_AVX_STREAM=${{ matrix.enable_kernel_avx_stream }} \ + -DLQ_ENABLE_KERNEL_OMP=${{ matrix.enable_kernel_omp }} cmake --build ./Build - cmake --build ./BuildKernelOMP - cmake --build ./BuildKernelAVXStream - - for d in Build BuildKernelOMP BuildKernelAVXStream; do - cd ./$d - mkdir -p ./tests/results - for file in *runner ; do ./$file --order lex --reporter junit --out ./tests/results/report_$file.xml; done; - lcov --directory . -b ../pennylane_lightning/core/src --capture --output-file coverage.info - lcov --remove coverage.info '/usr/*' --output-file coverage.info - cd .. - done - lcov --add-tracefile ./Build/coverage.info -a ./BuildKernelOMP/coverage.info \ - --add-tracefile ./BuildKernelAVXStream/coverage.info -o coverage.info - mv coverage.info coverage-${{ github.job }}-${{ matrix.pl_backend }}.info + + cd ./Build + mkdir -p ./tests/results + for file in *runner ; do ./$file --order lex --reporter junit --out ./tests/results/report_$file.xml; done; + lcov --directory . -b ../pennylane_lightning/core/src --capture --output-file coverage.info + lcov --remove coverage.info '/usr/*' --output-file coverage.info + mv coverage.info coverage-${{ github.job }}-${{ matrix.pl_backend }}-${{ matrix.enable_kernel_avx_stream }}-${{ matrix.enable_kernel_omp }}.info - name: Upload test results uses: actions/upload-artifact@v3 if: always() with: - name: ubuntu-tests-reports-${{ github.job }}-${{ matrix.pl_backend }} + name: ubuntu-tests-reports-${{ github.job }}-${{ matrix.pl_backend }}-${{ matrix.enable_kernel_avx_stream }}-${{ matrix.enable_kernel_omp }} path: | ./Build/tests/results/ - ./BuildKernelOMP/tests/results/ - ./BuildKernelAVXStream/tests/results/ if-no-files-found: error - name: Upload code coverage results uses: actions/upload-artifact@v3 with: - name: ubuntu-codecov-results-cpp - path: ./coverage-${{ github.job }}-${{ matrix.pl_backend }}.info + name: ubuntu-codecov-results-${{ matrix.pl_backend }}-${{ matrix.enable_kernel_avx_stream }}-${{ matrix.enable_kernel_omp }} + path: ./Build/coverage-${{ github.job }}-${{ matrix.pl_backend }}-${{ matrix.enable_kernel_avx_stream }}-${{ matrix.enable_kernel_omp }}.info if-no-files-found: error cpptestswithOpenBLAS: @@ -173,14 +152,14 @@ jobs: uses: actions/upload-artifact@v3 if: always() with: - name: ubuntu-tests-reports-${{ github.job }}-${{ matrix.pl_backend }} + name: ubuntu-tests-reports-blas-${{ github.job }}-${{ matrix.pl_backend }} path: ./Build/tests/results/ if-no-files-found: error - name: Upload code coverage results uses: actions/upload-artifact@v3 with: - name: ubuntu-codecov-results-cpp + name: ubuntu-codecov-results-blas-${{ matrix.pl_backend }} path: ./Build/coverage-${{ github.job }}-${{ matrix.pl_backend }}.info if-no-files-found: error @@ -245,8 +224,8 @@ jobs: -DENABLE_COVERAGE=ON cmake --build ./Build cd ./Build - mkdir -p ./tests/results_${{ github.job }}_${{ matrix.pl_backend }} - for file in *runner ; do ./$file --order lex --reporter junit --out ./tests/results_${{ github.job }}_${{ matrix.pl_backend }}/report_$file.xml; done; + mkdir -p ./tests/results-${{ github.job }}-${{ matrix.pl_backend }} + for file in *runner ; do ./$file --order lex --reporter junit --out ./tests/results-${{ github.job }}-${{ matrix.pl_backend }}/report_$file.xml; done; lcov --directory . -b ../pennylane_lightning/core/src --capture --output-file coverage.info lcov --remove coverage.info '/usr/*' --output-file coverage.info mv coverage.info coverage-${{ github.job }}-${{ matrix.pl_backend }}.info @@ -255,8 +234,8 @@ jobs: uses: actions/upload-artifact@v3 if: always() with: - name: ubuntu-tests-reports-${{ github.job }}_${{ matrix.pl_backend }} - path: ./Build/tests/results_${{ github.job }}_${{ matrix.pl_backend }} + name: ubuntu-tests-reports-${{ github.job }}-${{ matrix.pl_backend }} + path: ./Build/tests/results-${{ github.job }}-${{ matrix.pl_backend }} if-no-files-found: error - name: Upload code coverage results @@ -329,7 +308,6 @@ jobs: - name: Build and run unit tests run: | - set -x cmake . -BBuild -G Ninja \ -DCMAKE_BUILD_TYPE=Debug \ -DBUILD_TESTS=ON \ @@ -341,7 +319,6 @@ jobs: cd ./Build mkdir -p ./tests/results_multiple_backends for file in *runner ; do ./$file --order lex --reporter junit --out ./tests/results_multiple_backends/report_$file.xml; done; - set +x - name: Upload test results uses: actions/upload-artifact@v3 diff --git a/.github/workflows/tests_linux_python.yml b/.github/workflows/tests_linux_python.yml deleted file mode 100644 index dbb6ee1037..0000000000 --- a/.github/workflows/tests_linux_python.yml +++ /dev/null @@ -1,440 +0,0 @@ -name: Testing::x86_64::Python -on: - workflow_call: - inputs: - lightning-version: - type: string - required: true - description: The version of Lightning to use. Valid values are either 'release' (most recent release candidate), 'stable' (most recent git-tag) or 'latest' (most recent commit from master) - pennylane-version: - type: string - required: true - description: The version of PennyLane to use. Valid values are either 'release' (most recent release candidate), 'stable' (most recent git-tag) or 'latest' (most recent commit from master) - pull_request: - paths-ignore: - - pennylane_lightning/core/src/simulators/lightning_gpu/** - - pennylane_lightning/lightning_gpu/** - push: - branches: - - master - -env: - TF_VERSION: 2.10.0 - TORCH_VERSION: 1.11.0+cpu - COVERAGE_FLAGS: "--cov=pennylane_lightning --cov-report=term-missing --no-flaky-report -p no:warnings --tb=native" - GCC_VERSION: 11 - OMP_NUM_THREADS: "2" - OMP_PROC_BIND: "false" - -concurrency: - group: tests_linux_python-${{ github.ref }}-${{ github.event }}-${{ inputs.lightning-version }}-${{ inputs.pennylane-version }} - cancel-in-progress: true - -jobs: - determine_runner: - name: Determine runner type to use - uses: ./.github/workflows/determine-workflow-runner.yml - with: - default_runner: ubuntu-22.04 - - - pythontests: - needs: [determine_runner] - strategy: - matrix: - pl_backend: ["lightning_qubit"] - timeout-minutes: 75 - name: Python tests - runs-on: ${{ needs.determine_runner.outputs.runner_group }} - - steps: - - uses: actions/setup-python@v4 - name: Install Python - with: - python-version: '3.9' - - - name: Checkout PennyLane-Lightning - uses: actions/checkout@v4 - with: - fetch-tags: true - path: main - - - name: Switch to release build of Lightning - if: inputs.lightning-version == 'release' - run: | - cd main - git fetch --all - git checkout $(git branch -a --list "origin/v*rc*" | tail -1) - - - name: Switch to stable build of Lightning - if: inputs.lightning-version == 'stable' - run: | - cd main - git fetch --tags --force - git checkout $(git tag | sort -V | tail -1) - git log -1 --format='%H' - git status - - - uses: actions/setup-python@v5 - name: Install Python - with: - python-version: '3.9' - - - name: Install dependencies - run: | - sudo apt-get update && sudo apt-get -y -q install cmake gcc-$GCC_VERSION g++-$GCC_VERSION - python -m pip install scipy - - - name: Get required Python packages - run: | - cd main - python -m pip install -r requirements-dev.txt - python -m pip install openfermionpyscf - - - name: Checkout PennyLane for release build - if: inputs.pennylane-version == 'release' - uses: actions/checkout@v4 - with: - path: pennylane - repository: PennyLaneAI/pennylane - - - name: Switch to release build of PennyLane - if: inputs.pennylane-version == 'release' - run: | - cd pennylane - git fetch --all - git checkout $(git branch -a --list "origin/v*rc*" | tail -1) - python -m pip uninstall -y pennylane && python -m pip install . -vv --no-deps - - - name: Install Stable PennyLane - if: inputs.pennylane-version == 'stable' - run: | - cd main - python -m pip uninstall -y pennylane && python -m pip install -U pennylane - - - name: Install ML libraries for interfaces - run: | - python -m pip install --upgrade torch==$TORCH_VERSION -f https://download.pytorch.org/whl/cpu/torch_stable.html - python -m pip install --upgrade "jax[cpu]" # This also installs jaxlib - python -m pip install --upgrade tensorflow~=$TF_VERSION keras~=$TF_VERSION - - - name: Install backend device - run: | - cd main - CMAKE_ARGS="-DPL_BACKEND=${{ matrix.pl_backend }} -DLQ_ENABLE_KERNEL_OMP=ON -DENABLE_PYTHON=ON -DCMAKE_CXX_COMPILER=$(which g++-$GCC_VERSION)" \ - python -m pip install -e . -vv - - - name: Run PennyLane-Lightning unit tests - run: | - # TODO: Remove installing pytest-xdist with release v0.36.0 - python -m pip install pytest-xdist - cd main/ - DEVICENAME=`echo ${{ matrix.pl_backend }} | sed "s/_/./g"` - OMP_NUM_THREADS=1 PL_DEVICE=${DEVICENAME} python -m pytest -n auto tests/ -k "not unitary_correct" $COVERAGE_FLAGS - PL_DEVICE=${DEVICENAME} python -m pytest tests/ -k "unitary_correct" $COVERAGE_FLAGS --cov-append - pl-device-test --device ${DEVICENAME} --skip-ops --shots=20000 $COVERAGE_FLAGS --cov-append - pl-device-test --device ${DEVICENAME} --shots=None --skip-ops $COVERAGE_FLAGS --cov-append - mv .coverage .coverage-${{ github.job }}-${{ matrix.pl_backend }} - - - name: Upload code coverage results - uses: actions/upload-artifact@v3 - with: - name: ubuntu-codecov-results-python - path: | - ./main/.coverage-${{ github.job }}-${{ matrix.pl_backend }} - if-no-files-found: error - - - pythontestswithOpenBLAS: - needs: [determine_runner] - strategy: - matrix: - pl_backend: ["lightning_qubit"] - timeout-minutes: 75 - name: Python tests with OpenBLAS - runs-on: ${{ needs.determine_runner.outputs.runner_group }} - - steps: - - name: Checkout PennyLane-Lightning - uses: actions/checkout@v4 - with: - fetch-tags: true - path: main - - - name: Switch to release build of Lightning - if: inputs.lightning-version == 'release' - run: | - cd main - git fetch --all - git checkout $(git branch -a --list "origin/v*rc*" | tail -1) - - - name: Switch to stable build of Lightning - if: inputs.lightning-version == 'stable' - run: | - cd main - git fetch --tags --force - git checkout $(git tag | sort -V | tail -1) - git log -1 --format='%H' - git status - - - uses: actions/setup-python@v5 - name: Install Python - with: - python-version: '3.9' - - - name: Install dependencies - run: | - sudo apt-get update && sudo apt-get -y -q install cmake gcc-$GCC_VERSION g++-$GCC_VERSION libopenblas-dev - python -m pip install scipy - - - name: Get required Python packages - run: | - cd main - python -m pip install -r requirements-dev.txt - python -m pip install openfermionpyscf - - - name: Checkout PennyLane for release build - if: inputs.pennylane-version == 'release' - uses: actions/checkout@v4 - with: - path: pennylane - repository: PennyLaneAI/pennylane - - - name: Switch to release build of PennyLane - if: inputs.pennylane-version == 'release' - run: | - cd pennylane - git fetch --all - git checkout $(git branch -a --list "origin/v*rc*" | tail -1) - python -m pip uninstall -y pennylane && python -m pip install . -vv --no-deps - - - name: Install Stable PennyLane - if: inputs.pennylane-version == 'stable' - run: | - cd main - python -m pip uninstall -y pennylane && python -m pip install -U pennylane - - - name: Install ML libraries for interfaces - run: | - python -m pip install --upgrade torch==$TORCH_VERSION -f https://download.pytorch.org/whl/cpu/torch_stable.html - python -m pip install --upgrade "jax[cpu]" # This also installs jaxlib - python -m pip install --upgrade tensorflow~=$TF_VERSION keras~=$TF_VERSION - - - name: Install backend device - run: | - cd main - python -m pip install scipy - CMAKE_ARGS="-DPL_BACKEND=${{ matrix.pl_backend }} -DENABLE_BLAS=ON -DENABLE_PYTHON=ON -DCMAKE_CXX_COMPILER=$(which g++-$GCC_VERSION)" \ - python -m pip install -e . -vv - - - name: Run PennyLane-Lightning unit tests - run: | - # TODO: Remove installing pytest-xdist with release v0.36.0 - python -m pip install pytest-xdist - cd main/ - DEVICENAME=`echo ${{ matrix.pl_backend }} | sed "s/_/./g"` - OMP_NUM_THREADS=1 PL_DEVICE=${DEVICENAME} python -m pytest -n auto tests/ -k "not unitary_correct and not test_native_mcm" $COVERAGE_FLAGS - PL_DEVICE=${DEVICENAME} python -m pytest tests/ -k "unitary_correct and not test_native_mcm" $COVERAGE_FLAGS --cov-append - pl-device-test --device ${DEVICENAME} --skip-ops --shots=20000 $COVERAGE_FLAGS --cov-append - pl-device-test --device ${DEVICENAME} --shots=None --skip-ops $COVERAGE_FLAGS --cov-append - mv .coverage .coverage-${{ github.job }}-${{ matrix.pl_backend }} - - - name: Upload code coverage results - uses: actions/upload-artifact@v3 - with: - name: ubuntu-codecov-results-python - path: ./main/.coverage-${{ github.job }}-${{ matrix.pl_backend }} - if-no-files-found: error - - build_and_cache_Kokkos: - name: "Build and cache Kokkos" - needs: [determine_runner] - uses: ./.github/workflows/build_and_cache_Kokkos_linux.yml - with: - os: ubuntu-22.04 - runs_on: ${{ needs.determine_runner.outputs.runner_group }} - - pythontestswithKokkos: - needs: [build_and_cache_Kokkos, determine_runner] - strategy: - matrix: - os: [ubuntu-22.04] - pl_backend: ["lightning_kokkos", "all"] - exec_model: ${{ fromJson(needs.build_and_cache_Kokkos.outputs.exec_model) }} - kokkos_version: ${{ fromJson(needs.build_and_cache_Kokkos.outputs.kokkos_version) }} - exclude: - - pl_backend: ["all"] - exec_model: OPENMP - timeout-minutes: 75 - name: Python tests with Kokkos - runs-on: ${{ needs.determine_runner.outputs.runner_group }} - - steps: - - name: Checkout PennyLane-Lightning - uses: actions/checkout@v4 - with: - fetch-tags: true - path: main - - - name: Switch to release build of Lightning - if: inputs.lightning-version == 'release' - run: | - cd main - git fetch --all - git checkout $(git branch -a --list "origin/v*rc*" | tail -1) - - - name: Switch to stable build of Lightning - if: inputs.lightning-version == 'stable' - run: | - cd main - git fetch --tags --force - git checkout $(git tag | sort -V | tail -1) - git log -1 --format='%H' - git status - - - uses: actions/setup-python@v5 - name: Install Python - with: - python-version: '3.9' - - - name: Restoring cached dependencies - id: kokkos-cache - uses: actions/cache@v4 - with: - path: ${{ github.workspace}}/Kokkos_install/${{ matrix.exec_model }} - key: ${{ matrix.os }}-kokkos${{ matrix.kokkos_version }}-${{ matrix.exec_model }} - - - name: Copy cached libraries - if: steps.kokkos-cache.outputs.cache-hit == 'true' - run: | - rm -rf Kokkos - mkdir Kokkos/ - cp -rf ${{ github.workspace}}/Kokkos_install/${{ matrix.exec_model }}/* Kokkos/ - pwd - - - name: Install dependencies - run: | - sudo apt-get update && sudo apt-get -y -q install cmake gcc-$GCC_VERSION g++-$GCC_VERSION - python -m pip install scipy - - - name: Get required Python packages - run: | - cd main - python -m pip install -r requirements-dev.txt - python -m pip install openfermionpyscf - - - name: Checkout PennyLane for release build - if: inputs.pennylane-version == 'release' - uses: actions/checkout@v4 - with: - path: pennylane - repository: PennyLaneAI/pennylane - - - name: Switch to release build of PennyLane - if: inputs.pennylane-version == 'release' - run: | - cd pennylane - git fetch --all - git checkout $(git branch -a --list "origin/v*rc*" | tail -1) - python -m pip uninstall -y pennylane && python -m pip install . -vv --no-deps - - - name: Install Stable PennyLane - if: inputs.pennylane-version == 'stable' - run: | - cd main - python -m pip uninstall -y pennylane && python -m pip install -U pennylane - - - name: Install ML libraries for interfaces - run: | - python -m pip install --upgrade torch==$TORCH_VERSION -f https://download.pytorch.org/whl/cpu/torch_stable.html - python -m pip install --upgrade "jax[cpu]" # This also installs jaxlib - python -m pip install --upgrade tensorflow~=$TF_VERSION keras~=$TF_VERSION - - - name: Install backend device - if: ${{ matrix.pl_backend != 'all'}} - run: | - cd main - python -m pip install scipy - SKIP_COMPILATION=True PL_BACKEND="lightning_qubit" pip install -e . -vv - CMAKE_ARGS="-DPL_BACKEND=${{ matrix.pl_backend }} -DCMAKE_PREFIX_PATH=${{ github.workspace }}/Kokkos -DENABLE_PYTHON=ON -DCMAKE_CXX_COMPILER=$(which g++-$GCC_VERSION)" \ - python -m pip install -e . -vv - - - name: Run PennyLane-Lightning unit tests - if: ${{ matrix.pl_backend != 'all'}} - run: | - # TODO: Remove installing pytest-xdist with release v0.36.0 - python -m pip install pytest-xdist - cd main/ - DEVICENAME=`echo ${{ matrix.pl_backend }} | sed "s/_/./g"` - PL_DEVICE=${DEVICENAME} python -m pytest tests/ -k "not test_native_mcm" $COVERAGE_FLAGS - pl-device-test --device ${DEVICENAME} --skip-ops --shots=20000 $COVERAGE_FLAGS --cov-append - pl-device-test --device ${DEVICENAME} --shots=None --skip-ops $COVERAGE_FLAGS --cov-append - mv .coverage .coverage-${{ github.job }}-${{ matrix.pl_backend }} - # TODO: Remove this if-cond with release v0.36.0 - if [ -f tests/test_native_mcm.py ]; then - OMP_NUM_THREADS=1 PL_DEVICE=${DEVICENAME} python -m pytest -n auto tests/ -k "test_native_mcm" $COVERAGE_FLAGS --cov-append - fi - - - name: Install all backend devices - if: ${{ matrix.pl_backend == 'all' }} - run: | - cd main - CMAKE_ARGS="-DPL_BACKEND="lightning_qubit" -DCMAKE_PREFIX_PATH=${{ github.workspace }}/Kokkos -DENABLE_PYTHON=ON -DCMAKE_CXX_COMPILER=$(which g++-$GCC_VERSION)" \ - python -m pip install -e . -vv - rm -rf build - CMAKE_ARGS="-DPL_BACKEND="lightning_kokkos" -DCMAKE_PREFIX_PATH=${{ github.workspace }}/Kokkos -DENABLE_PYTHON=ON -DCMAKE_CXX_COMPILER=$(which g++-$GCC_VERSION)" \ - python -m pip install -e . -vv - - - name: Run PennyLane-Lightning unit tests for lightning.qubit with all devices installed - if: ${{ matrix.pl_backend == 'all' }} - run: | - # TODO: Remove installing pytest-xdist with release v0.36.0 - python -m pip install pytest-xdist - cd main/ - OMP_NUM_THREADS=1 PL_DEVICE=lightning.qubit python -m pytest -n auto tests/ -k "not unitary_correct and not test_native_mcm" $COVERAGE_FLAGS - PL_DEVICE=lightning.qubit python -m pytest tests/ -k "unitary_correct and not test_native_mcm" $COVERAGE_FLAGS --cov-append - pl-device-test --device lightning.qubit --skip-ops --shots=20000 $COVERAGE_FLAGS --cov-append - pl-device-test --device lightning.qubit --shots=None --skip-ops $COVERAGE_FLAGS --cov-append - PL_DEVICE=lightning.kokkos python -m pytest tests/ -k "not test_native_mcm" $COVERAGE_FLAGS --cov-append - pl-device-test --device lightning.kokkos --skip-ops --shots=20000 $COVERAGE_FLAGS --cov-append - pl-device-test --device lightning.kokkos --shots=None --skip-ops $COVERAGE_FLAGS --cov-append - mv .coverage .coverage-${{ github.job }}-${{ matrix.pl_backend }} - - - name: Upload code coverage results - uses: actions/upload-artifact@v3 - with: - name: ubuntu-codecov-results-python - path: ./main/.coverage-${{ github.job }}-${{ matrix.pl_backend }} - if-no-files-found: error - - upload-to-codecov-linux-python: - needs: [pythontests, pythontestswithOpenBLAS, pythontestswithKokkos] - name: Upload python coverage data to codecov - runs-on: ubuntu-latest - steps: - - name: Checkout - uses: actions/checkout@v4 - with: - fetch-tags: true - path: main - - - name: Download coverage reports - uses: actions/download-artifact@v3 - with: - name: ubuntu-codecov-results-python - - - name: Combine coverage files - run: | - python -m pip install coverage - python -m coverage combine .coverage-python* - # Added cov xml -i to ignore "No source for code" random errors - # https://stackoverflow.com/questions/2386975/no-source-for-code-message-in-coverage-py - python -m coverage xml -i -o coverage-${{ github.job }}.xml - - - name: Upload to Codecov - uses: codecov/codecov-action@v4 - with: - fail_ci_if_error: true - verbose: true - token: ${{ secrets.CODECOV_TOKEN }} - diff --git a/.github/workflows/tests_lkcpu_python.yml b/.github/workflows/tests_lkcpu_python.yml index e44c08db67..5d405753c7 100644 --- a/.github/workflows/tests_lkcpu_python.yml +++ b/.github/workflows/tests_lkcpu_python.yml @@ -12,6 +12,8 @@ on: description: The version of PennyLane to use. Valid values are either 'release' (most recent release candidate), 'stable' (most recent git-tag) or 'latest' (most recent commit from master) pull_request: paths-ignore: + - .github + - '!.github/workflows/tests_lkcpu_python.yml' - pennylane_lightning/core/src/simulators/lightning_gpu/** - pennylane_lightning/lightning_gpu/** - pennylane_lightning/core/src/simulators/lightning_qubit/** @@ -34,6 +36,12 @@ concurrency: jobs: + determine_runner: + name: Determine runner type to use + uses: ./.github/workflows/determine-workflow-runner.yml + with: + default_runner: ubuntu-22.04 + build_and_cache_Kokkos: name: "Build and cache Kokkos" uses: ./.github/workflows/build_and_cache_Kokkos_linux.yml @@ -41,7 +49,7 @@ jobs: os: ubuntu-22.04 build_lightning_kokkos_wheels: - needs: [build_and_cache_Kokkos] + needs: [determine_runner, build_and_cache_Kokkos] strategy: matrix: os: [ubuntu-22.04] @@ -50,7 +58,7 @@ jobs: kokkos_version: ${{ fromJson(needs.build_and_cache_Kokkos.outputs.kokkos_version) }} timeout-minutes: 60 name: Python tests with Kokkos - runs-on: ${{ matrix.os }} + runs-on: ${{ needs.determine_runner.outputs.runner_group }} steps: - name: Checkout PennyLane-Lightning @@ -135,7 +143,7 @@ jobs: ${{ github.workspace }}/wheel_${{ matrix.pl_backend }}-${{ matrix.exec_model }}.whl test_lightning_kokkos_wheels: - needs: [build_and_cache_Kokkos, build_lightning_kokkos_wheels] + needs: [determine_runner, build_and_cache_Kokkos, build_lightning_kokkos_wheels] strategy: matrix: os: [ubuntu-22.04] @@ -145,7 +153,7 @@ jobs: kokkos_version: ${{ fromJson(needs.build_and_cache_Kokkos.outputs.kokkos_version) }} timeout-minutes: 60 name: Python tests with Kokkos - runs-on: ${{ matrix.os }} + runs-on: ${{ needs.determine_runner.outputs.runner_group }} steps: - uses: actions/setup-python@v5 @@ -235,9 +243,9 @@ jobs: if-no-files-found: error upload-to-codecov-linux-python: - needs: [test_lightning_kokkos_wheels] + needs: [determine_runner, test_lightning_kokkos_wheels] name: Upload python coverage data to codecov - runs-on: ubuntu-latest + runs-on: ${{ needs.determine_runner.outputs.runner_group }} steps: - name: Checkout uses: actions/checkout@v4 diff --git a/.github/workflows/tests_lkcuda_python.yml b/.github/workflows/tests_lkcuda_python.yml index 4092080efa..a7daef4cf1 100644 --- a/.github/workflows/tests_lkcuda_python.yml +++ b/.github/workflows/tests_lkcuda_python.yml @@ -12,6 +12,8 @@ on: description: The version of PennyLane to use. Valid values are either 'release' (most recent release candidate), 'stable' (most recent git-tag) or 'latest' (most recent commit from master) pull_request: paths-ignore: + - .github + - '!.github/workflows/tests_lkcuda_python.yml' - pennylane_lightning/core/src/simulators/lightning_gpu/** - pennylane_lightning/core/src/simulators/lightning_qubit/** - pennylane_lightning/lightning_gpu/** diff --git a/.github/workflows/tests_lqcpu_python.yml b/.github/workflows/tests_lqcpu_python.yml index 5c48006936..89a3869b98 100644 --- a/.github/workflows/tests_lqcpu_python.yml +++ b/.github/workflows/tests_lqcpu_python.yml @@ -12,6 +12,8 @@ on: description: The version of PennyLane to use. Valid values are either 'release' (most recent release candidate), 'stable' (most recent git-tag) or 'latest' (most recent commit from master) pull_request: paths-ignore: + - .github + - '!.github/workflows/tests_lqcpu_python.yml' - pennylane_lightning/core/src/simulators/lightning_gpu/** - pennylane_lightning/lightning_gpu/** - pennylane_lightning/core/src/simulators/lightning_kokkos/** @@ -33,15 +35,21 @@ concurrency: cancel-in-progress: true jobs: + determine_runner: + name: Determine runner type to use + uses: ./.github/workflows/determine-workflow-runner.yml + with: + default_runner: ubuntu-22.04 + build_lightning_qubit_wheels: + needs: [determine_runner] strategy: matrix: - os: [ubuntu-22.04] pl_backend: ["lightning_qubit"] blas: ["OFF", "ON"] timeout-minutes: 60 name: Python tests - runs-on: ${{ matrix.os }} + runs-on: ${{ needs.determine_runner.outputs.runner_group }} steps: - uses: actions/setup-python@v5 @@ -110,16 +118,15 @@ jobs: ${{ github.workspace }}/wheel_${{ matrix.pl_backend }}-${{ matrix.blas }}.whl test_lightning_qubit_wheels: - needs: ["build_lightning_qubit_wheels"] + needs: [determine_runner, build_lightning_qubit_wheels] strategy: matrix: - os: [ubuntu-22.04] pl_backend: ["lightning_qubit"] blas: ["OFF", "ON"] group: [1, 2, 3, 4] timeout-minutes: 60 name: Python tests - runs-on: ${{ matrix.os }} + runs-on: ${{ needs.determine_runner.outputs.runner_group }} steps: - uses: actions/setup-python@v5 @@ -202,9 +209,9 @@ jobs: if-no-files-found: error upload-to-codecov-linux-python: - needs: [test_lightning_qubit_wheels] + needs: [determine_runner, test_lightning_qubit_wheels] name: Upload python coverage data to codecov - runs-on: ubuntu-latest + runs-on: ${{ needs.determine_runner.outputs.runner_group }} steps: - name: Checkout uses: actions/checkout@v4 diff --git a/.github/workflows/tidy.yml b/.github/workflows/tidy.yml index 64cf7247d6..68e9dd1a9c 100644 --- a/.github/workflows/tidy.yml +++ b/.github/workflows/tidy.yml @@ -19,11 +19,10 @@ jobs: tidy-cpp: strategy: matrix: - os: [ubuntu-22.04] pl_backend: ["lightning_qubit"] name: Tidy (C++) - runs-on: ${{ matrix.os }} + runs-on: ubuntu-22.04 steps: - name: Checkout PennyLane-Lightning