Add BotorchTestCase.assertAllClose #1618
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
facebook-github-bot
added
CLA Signed
|
This pull request was exported from Phabricator. Differential Revision: D42402142 |
Codecov Report
@@ Coverage Diff @@
## main #1618 +/- ##
=========================================
Coverage 100.00% 100.00%
=========================================
Files 154 154
Lines 13774 13776 +2
=========================================
+ Hits 13774 13776 +2
📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
Summary: Pull Request resolved: pytorch#1618 `BotorchTestCase.assertAllClose` will print more informative error messages on failure than `TestCase.assertTrue(torch.allclose(...))`. It uses `torch.testing.assert_close`. Old test output: ```AssertionError: False is not true``` New test output: ``` 1) AssertionError: Scalars are not close! Absolute difference: 1.0000034868717194 (up to 0.0001 allowed) Relative difference: 0.8348668001940709 (up to 1e-05 allowed) ``` This currently replicates the behavior of `torch.allclose` so that tests remain exactly as strict as they used to be, but in the future we might want to use the behavior of `assert_close` instead since it uses higher tolerances for single-precision inputs by default and is more configurable. Differential Revision: D42402142 fbshipit-source-id: 62dc6df5e786a72a758a4d9a08ca92f88ab2c149
esantorella
force-pushed
the
export-D42402142
branch
from
11c7228 to
19ec6e1
Compare
|
This pull request was exported from Phabricator. Differential Revision: D42402142 |
esantorella
added a commit
to esantorella/botorch
that referenced
this pull request
Jan 7, 2023
Summary: Pull Request resolved: pytorch#1618 `BotorchTestCase.assertAllClose` is a thin wrapper around `torch.testing.assert_close`, designed to replace usages of `self.assertTrue(torch.allclose(...))`. Using has several advantages over `torch.allclose`: * Checks that shapes are equal, not just values * More configurability and better defaults, such as higher tolerances for single precision, if we choose to use them in the future. for the time being, I've set up this wrapper so that numerical checks remain exactly the same as they used to be. * More informative test output, showing what was put in and why the test failed: Old test output: ```AssertionError: False is not true``` New test output: ``` 1) AssertionError: Scalars are not close! Absolute difference: 1.0000034868717194 (up to 0.0001 allowed) Relative difference: 0.8348668001940709 (up to 1e-05 allowed) ``` Differential Revision: https://internalfb.com/D42402142 fbshipit-source-id: c729ed6ea7c0e16487e3bc7e174b6e4c675cc19a
esantorella
added a commit
to esantorella/botorch
that referenced
this pull request
Jan 7, 2023
Summary: Pull Request resolved: pytorch#1618 `BotorchTestCase.assertAllClose` is a thin wrapper around `torch.testing.assert_close`, designed to replace usages of `self.assertTrue(torch.allclose(...))`. Using has several advantages over `torch.allclose`: * Checks that shapes are equal, not just values * More configurability and better defaults, such as higher tolerances for single precision, if we choose to use them in the future. for the time being, I've set up this wrapper so that numerical checks remain exactly the same as they used to be. * More informative test output, showing what was put in and why the test failed: Old test output: ```AssertionError: False is not true``` New test output: ``` 1) AssertionError: Scalars are not close! Absolute difference: 1.0000034868717194 (up to 0.0001 allowed) Relative difference: 0.8348668001940709 (up to 1e-05 allowed) ``` Differential Revision: https://internalfb.com/D42402142 fbshipit-source-id: 1bec649658ea698a7f54330d20837efa0d2aaa11
esantorella
added a commit
to esantorella/botorch
that referenced
this pull request
Jan 7, 2023
Summary: Pull Request resolved: pytorch#1618 `BotorchTestCase.assertAllClose` is a thin wrapper around `torch.testing.assert_close`, designed to replace usages of `self.assertTrue(torch.allclose(...))`. Using has several advantages over `torch.allclose`: * Checks that shapes are equal, not just values * More configurability and better defaults, such as higher tolerances for single precision, if we choose to use them in the future. for the time being, I've set up this wrapper so that numerical checks remain exactly the same as they used to be. * More informative test output, showing what was put in and why the test failed: Old test output: ```AssertionError: False is not true``` New test output: ``` 1) AssertionError: Scalars are not close! Absolute difference: 1.0000034868717194 (up to 0.0001 allowed) Relative difference: 0.8348668001940709 (up to 1e-05 allowed) ``` Differential Revision: https://internalfb.com/D42402142 fbshipit-source-id: 24df18d1e748f4b4e9192f3781dee75259bbb80c
esantorella
added a commit
to esantorella/botorch
that referenced
this pull request
Jan 7, 2023
Summary: Pull Request resolved: pytorch#1618 `BotorchTestCase.assertAllClose` is a thin wrapper around `torch.testing.assert_close`, designed to replace usages of `self.assertTrue(torch.allclose(...))`. Using has several advantages over `torch.allclose`: * Checks that shapes are equal, not just values * More configurability and better defaults, such as higher tolerances for single precision, if we choose to use them in the future. for the time being, I've set up this wrapper so that numerical checks remain exactly the same as they used to be. * More informative test output, showing what was put in and why the test failed: Old test output: ```AssertionError: False is not true``` New test output: ``` 1) AssertionError: Scalars are not close! Absolute difference: 1.0000034868717194 (up to 0.0001 allowed) Relative difference: 0.8348668001940709 (up to 1e-05 allowed) ``` Differential Revision: https://internalfb.com/D42402142 fbshipit-source-id: e10063d1b3401168e6e7bfd1fc605b2331a63a3d
esantorella
added a commit
to esantorella/botorch
that referenced
this pull request
Jan 7, 2023
Summary: Pull Request resolved: pytorch#1618 `BotorchTestCase.assertAllClose` is a thin wrapper around `torch.testing.assert_close`, designed to replace usages of `self.assertTrue(torch.allclose(...))`. Using has several advantages over `torch.allclose`: * Checks that shapes are equal, not just values * More configurability and better defaults, such as higher tolerances for single precision, if we choose to use them in the future. for the time being, I've set up this wrapper so that numerical checks remain exactly the same as they used to be. * More informative test output, showing what was put in and why the test failed: Old test output: ```AssertionError: False is not true``` New test output: ``` 1) AssertionError: Scalars are not close! Absolute difference: 1.0000034868717194 (up to 0.0001 allowed) Relative difference: 0.8348668001940709 (up to 1e-05 allowed) ``` Differential Revision: https://internalfb.com/D42402142 fbshipit-source-id: 3ceb5a02ee6704da4a49cc48f5bb7adf7b2e9c9a
|
This pull request has been merged in 05d93e6. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
BotorchTestCase.assertAllClosewill print more informative error messages on failure thanTestCase.assertTrue(torch.allclose(...)). It usestorch.testing.assert_close.Old test output:
AssertionError: False is not trueNew test output:
This currently replicates the behavior of
torch.allcloseso that tests remain exactly as strict as they used to be, but in the future we might want to use the behavior ofassert_closeinstead since it uses higher tolerances for single-precision inputs by default and is more configurable.Differential Revision: D42402142