Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix pulse parameter value formatter bug #11972

Merged
merged 3 commits into from
Mar 14, 2024

Conversation

nkanazawa1989
Copy link
Contributor

Summary

This PR fix the pulse parameter assignment bug that happens with input of particular integer number.

Fix #11971

Details and comments

@nkanazawa1989 nkanazawa1989 added stable backport potential The bug might be minimal and/or import enough to be port to stable Changelog: Bugfix Include in the "Fixed" section of the changelog labels Mar 8, 2024
@nkanazawa1989 nkanazawa1989 requested review from eggerdj, wshanks and a team as code owners March 8, 2024 09:30
@qiskit-bot
Copy link
Collaborator

One or more of the the following people are requested to review this:

  • @Qiskit/terra-core
  • @nkanazawa1989

@coveralls
Copy link

coveralls commented Mar 8, 2024

Pull Request Test Coverage Report for Build 8283966462

Details

  • 15 of 15 (100.0%) changed or added relevant lines in 1 file are covered.
  • 15 unchanged lines in 2 files lost coverage.
  • Overall coverage increased (+0.01%) to 89.263%

Files with Coverage Reduction New Missed Lines %
crates/qasm2/src/lex.rs 3 91.94%
crates/qasm2/src/parse.rs 12 97.15%
Totals Coverage Status
Change from base Build 8282173373: 0.01%
Covered Lines: 59574
Relevant Lines: 66740

💛 - Coveralls

@nkanazawa1989 nkanazawa1989 removed the stable backport potential The bug might be minimal and/or import enough to be port to stable label Mar 8, 2024
@nkanazawa1989 nkanazawa1989 added this to the 1.1.0 milestone Mar 8, 2024
@jakelishman jakelishman added the mod: pulse Related to the Pulse module label Mar 8, 2024
Copy link
Contributor

@wshanks wshanks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good. I had a few questions. I think the issue only comes into play in practice for very large integers (the case leading to #11971 was caused by numbers not being multiplied by dt (2e-10) so they were much larger than expected), so it is not too urgent.

# Unassigned expression
return operand

if isinstance(operand, int):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would add a comment here that ints are being returned early to avoid rounding errors in the code below.

I was trying to understand what was wrong with the old code. np.round's documentation says that it effectively multiplies by 10**decimal (probably casting to float), rounds to an integer, and divides by 10**decimal. For a large enough integer, this operation must sometimes introduce rounding error in the float operations that causes is_integer in the old code to fail.

I was surprised because I was expecting your test cases to be around the 64 bit floating point precision of 1e19 but they are around 1e12. It kind of makes sense to me if there is an extra 1e10 in the operations.

Comment on lines +86 to +87
if evaluated.is_integer():
return int(evaluated)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think this is still important? In the old code, the number was always converted from int to float so this was needed to convert the input ints back into ints. With your changes we now know that the input was not an int so it might be okay to leave it as a float. Maybe some users rely on the casting. is_integer seems fragile combined with round though. Something that tests for almost an integer might be better.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. If the expression is something like 1.00000000001 (could happen if we numerically compute pulse duration), this object is the float type in early type check, and rounded to an integer and typecasted here. This is covered by the unit tests.

Copy link
Contributor Author

@nkanazawa1989 nkanazawa1989 Mar 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably very large integer with tiny rounding error might be worth testing. A number with many significant digits is rounded anyways.

Copy link
Contributor

@wshanks wshanks Mar 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to check -- do you think is_integer is good enough here or should it be

Suggested change
if evaluated.is_integer():
return int(evaluated)
if np.isclose(evaluated % 1, 0):
return int(evaluated)

to account for rounding error more leniently? Part of the original problem was that np.round did not always return something that would pass is_integer for an integer input.

One concern I have with this is that I think numpy.round could introduce floating point error that makes the number slightly below the nearest integer and then int(evaluated) could round down to the next integer. I guess it could be return int(evaluated + 0.1) to be safe though that is ugly.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's neat. Updated in 8ed6192. Thanks!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did some testing because I was still worried about rounding errors. What I found in practice was that is_integer always matched numpy.isclose(val % 1, 0), so numpy round must reliably round up above the nearest integer?

I undid your most recent change since is_integer seems clearer if there is no computational benefit to the modulus method.

Here is my script for reference. I found that casting to float, rounding, and casting back consistently matched the original number for numbers below 2**52, which makes sense since that is the precision of the significand in a 64 bit float. I got some surprising results for the fraction of results that did not match the input but that might be because my intuition was off for random numbers and exponential ranges.

import numpy


bad_counts = 0
min_bad = float("inf")
max_err = 0
max_nums = 100_000
total = 0
decimals = 1
max_int = 2**62
max_num = 0
for idx in range(10_000):
    start = numpy.random.randint(0, max_int, size=(10_000,))
    cast = start.astype(numpy.complex128)
    rounded = numpy.round(cast, decimals=decimals).astype(float)
    quotient, remainder = numpy.divmod(rounded, 1)
    # print(f"Round {idx}")
    int_checks = numpy.isclose(remainder, 0)
    # print(f"All close to ints still: {numpy.all(int_checks)}")
    final = quotient.astype(int)
    changed = start != final
    py_is_int = numpy.array([float(f).is_integer() for f in rounded])
    print(f"Not is_int:   {sum(numpy.logical_not(py_is_int))}")
    print(f"Not is_close: {sum(numpy.logical_not(int_checks))}")
    if not numpy.all(py_is_int == int_checks):
        print("is_integer differs from divmod")
    # print(f"int cast is equal: {numpy.all(changed)}")
    if numpy.all(int_checks) and numpy.any(changed):
        for start_i, final_i in zip(start[changed], final[changed]):

            pulse = int(numpy.real(numpy.round(complex(start_i), decimals=decimals)).item())
            # print("Bad input:")
            # print(f"Input:          {start_i}")
            # print(f"Output (numpy): {final_i}")
            # print(f"Output (pulse): {pulse}")
            if pulse != final_i:
                print(f"Pulse diff: {final_i}, {pulse}")
                break
        bad_counts += len(changed)
        min_bad = min(min_bad, min(start[changed]))
        max_err = max(max_err, max(abs(final[changed] - start[changed])))
    max_num = max(max_num, max(start))

    total += len(start)
    if total >= max_nums:
        break

print(f"Minimum bad input: {min_bad} ({min_bad:0.4g}), log2={numpy.log2(min_bad)}")
print(f"Maximum error: {max_err}")
print(f"Bad counts: {bad_counts} / {total}")
print(f"Maximum number processed: {max_num} ({max_num:0.4g}), log2={numpy.log2(max_num)}")

test/python/pulse/test_parameter_manager.py Show resolved Hide resolved
Co-authored-by: Will Shanks <wshaos@posteo.net>
qiskit/pulse/utils.py Outdated Show resolved Hide resolved
Copy link
Contributor

@wshanks wshanks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@wshanks wshanks enabled auto-merge March 14, 2024 16:19
@wshanks wshanks added this pull request to the merge queue Mar 14, 2024
Merged via the queue into Qiskit:main with commit f48d819 Mar 14, 2024
14 checks passed
@1ucian0 1ucian0 added the affects extended support This issue (also) affects extended support label May 30, 2024
@1ucian0
Copy link
Member

1ucian0 commented May 30, 2024

@Mergifyio backport stable/0.46

Copy link
Contributor

mergify bot commented May 30, 2024

backport stable/0.46

✅ Backports have been created

mergify bot pushed a commit that referenced this pull request May 30, 2024
* Fix value formatter bug

* reno

* Comments from review

Co-authored-by: Will Shanks <wshaos@posteo.net>

---------

Co-authored-by: Will Shanks <wshaos@posteo.net>
(cherry picked from commit f48d819)

# Conflicts:
#	test/python/pulse/test_parameter_manager.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
affects extended support This issue (also) affects extended support Changelog: Bugfix Include in the "Fixed" section of the changelog mod: pulse Related to the Pulse module
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Pulse parameter assignment bug
6 participants