Fix unnecessary serialisation of PassManager in serial contexts
#12410
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This exposes the interal decision in `parallel_map` of whether to actually run in serial or not. If not, there's no need for `PassManager` to side-car its `dill` serialisation onto the side of the IPC (we use `dill` because we need to pickle lambdas), which can be an unfortunately huge cost for certain IBM pulse-enabled backends.
jakelishman
added
Changelog: New Feature
|
One or more of the the following people are requested to review this:
|
mtreinish
reviewed
May 15, 2024
There was a problem hiding this comment.
To make this eligible for backport I was thinking that we leave should_run_in_parallel as an internal function for now. I agree having it in the public api makes sense longer term, but if we wanted to include this fix in 1.1.x we don't need to expose it in this PR.
Pull Request Test Coverage Report for Build 9098821180Details
💛 - Coveralls |
This makes the patch series safe for backport to 1.1.
jakelishman
added
stable backport potential
mtreinish
approved these changes
May 15, 2024
mergify bot
pushed a commit
that referenced
this pull request
May 15, 2024
…2410) * Fix unnecessary serialisation of `PassManager` in serial contexts This exposes the interal decision in `parallel_map` of whether to actually run in serial or not. If not, there's no need for `PassManager` to side-car its `dill` serialisation onto the side of the IPC (we use `dill` because we need to pickle lambdas), which can be an unfortunately huge cost for certain IBM pulse-enabled backends. * Remove new function from public API This makes the patch series safe for backport to 1.1. (cherry picked from commit b12e9ec)
github-merge-queue bot
pushed a commit
that referenced
this pull request
May 15, 2024
…2410) (#12413) * Fix unnecessary serialisation of `PassManager` in serial contexts This exposes the interal decision in `parallel_map` of whether to actually run in serial or not. If not, there's no need for `PassManager` to side-car its `dill` serialisation onto the side of the IPC (we use `dill` because we need to pickle lambdas), which can be an unfortunately huge cost for certain IBM pulse-enabled backends. * Remove new function from public API This makes the patch series safe for backport to 1.1. (cherry picked from commit b12e9ec) Co-authored-by: Jake Lishman <jake.lishman@ibm.com>
4 tasks
1ucian0
added
the
affects extended support
|
Do you think it makes sense to backport this to stable/0.46? |
ElePT
pushed a commit
to ElePT/qiskit
that referenced
this pull request
May 31, 2024
…skit#12410) * Fix unnecessary serialisation of `PassManager` in serial contexts This exposes the interal decision in `parallel_map` of whether to actually run in serial or not. If not, there's no need for `PassManager` to side-car its `dill` serialisation onto the side of the IPC (we use `dill` because we need to pickle lambdas), which can be an unfortunately huge cost for certain IBM pulse-enabled backends. * Remove new function from public API This makes the patch series safe for backport to 1.1.
|
@Mergifyio backport stable/0.46 |
✅ Backports have been created
|
mergify bot
pushed a commit
that referenced
this pull request
Jun 4, 2024
…2410) * Fix unnecessary serialisation of `PassManager` in serial contexts This exposes the interal decision in `parallel_map` of whether to actually run in serial or not. If not, there's no need for `PassManager` to side-car its `dill` serialisation onto the side of the IPC (we use `dill` because we need to pickle lambdas), which can be an unfortunately huge cost for certain IBM pulse-enabled backends. * Remove new function from public API This makes the patch series safe for backport to 1.1. (cherry picked from commit b12e9ec) # Conflicts: # qiskit/utils/__init__.py # qiskit/utils/parallel.py
github-merge-queue bot
pushed a commit
that referenced
this pull request
Jun 8, 2024
…ckport #12410) (#12500) * Fix unnecessary serialisation of `PassManager` in serial contexts (#12410) * Fix unnecessary serialisation of `PassManager` in serial contexts This exposes the interal decision in `parallel_map` of whether to actually run in serial or not. If not, there's no need for `PassManager` to side-car its `dill` serialisation onto the side of the IPC (we use `dill` because we need to pickle lambdas), which can be an unfortunately huge cost for certain IBM pulse-enabled backends. * Remove new function from public API This makes the patch series safe for backport to 1.1. (cherry picked from commit b12e9ec) # Conflicts: # qiskit/utils/__init__.py # qiskit/utils/parallel.py * solve conflicts * passmanager does not know the cpu_count --------- Co-authored-by: Jake Lishman <jake.lishman@ibm.com> Co-authored-by: Luciano Bello <bel@zurich.ibm.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This exposes the interal decision in
parallel_mapof whether to actually run in serial or not. If not, there's no need forPassManagerto side-car itsdillserialisation onto the side of the IPC (we usedillbecause we need to pickle lambdas), which can be an unfortunately huge cost for certain IBM pulse-enabled backends.Details and comments
@sbrandhsn: as the release manager for 1.1.0, it's up to you whether you want to accept this PR in this form for 1.1.0, or defer to 1.1.1 (you can tag the milestone appropriately). I could also suppress exporting
should_run_in_parallelinto the public namespace if you're not on board with a new public feature this late into the release cycle.