fatmax and logsumexp for infinities
#1999
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
facebook-github-bot
added
the
CLA Signed
|
This pull request was exported from Phabricator. Differential Revision: D48878020 |
SebastianAment
added a commit
to SebastianAment/botorch
that referenced
this pull request
Sep 1, 2023
Summary: This commit improves the robustness of `fatmax` and `logsumexp` for inputs with infinities. - In constrast to `torch.logsumexp`, `logsumexp` does not give rise to `NaN`s in its backward pass even if infinities are present. - `fatmax` is updated to exhibit the same behavior in the presence of infinities, and now allows for the specification of an `alpha` parameter, which controls the the asymptotic power decay of the fat-tailed approximation. In addition, the commit introduces helper functions derivative of `logsumexp` and `fatmax`, e.g. `logplusexp`, `fatminimum`, `fatmaximum`, fixes a similar infinity issue with `logdiffexp`, and improves the associated test suite. Reviewed By: Balandat Differential Revision: D48878020
SebastianAment
force-pushed
the
export-D48878020
branch
from
750170b to
08a965b
Compare
|
This pull request was exported from Phabricator. Differential Revision: D48878020 |
SebastianAment
added a commit
to SebastianAment/botorch
that referenced
this pull request
Sep 1, 2023
Summary: This commit improves the robustness of `fatmax` and `logsumexp` for inputs with infinities. - In constrast to `torch.logsumexp`, `logsumexp` does not give rise to `NaN`s in its backward pass even if infinities are present. - `fatmax` is updated to exhibit the same behavior in the presence of infinities, and now allows for the specification of an `alpha` parameter, which controls the the asymptotic power decay of the fat-tailed approximation. In addition, the commit introduces helper functions derivative of `logsumexp` and `fatmax`, e.g. `logplusexp`, `fatminimum`, `fatmaximum`, fixes a similar infinity issue with `logdiffexp`, and improves the associated test suite. Reviewed By: Balandat Differential Revision: D48878020
SebastianAment
force-pushed
the
export-D48878020
branch
from
08a965b to
55392c1
Compare
|
This pull request was exported from Phabricator. Differential Revision: D48878020 |
SebastianAment
added a commit
to SebastianAment/botorch
that referenced
this pull request
Sep 1, 2023
Summary: This commit improves the robustness of `fatmax` and `logsumexp` for inputs with infinities. - In constrast to `torch.logsumexp`, `logsumexp` does not give rise to `NaN`s in its backward pass even if infinities are present. - `fatmax` is updated to exhibit the same behavior in the presence of infinities, and now allows for the specification of an `alpha` parameter, which controls the the asymptotic power decay of the fat-tailed approximation. In addition, the commit introduces helper functions derivative of `logsumexp` and `fatmax`, e.g. `logplusexp`, `fatminimum`, `fatmaximum`, fixes a similar infinity issue with `logdiffexp`, and improves the associated test suite. Reviewed By: Balandat Differential Revision: D48878020
SebastianAment
force-pushed
the
export-D48878020
branch
from
55392c1 to
fec94c5
Compare
|
This pull request was exported from Phabricator. Differential Revision: D48878020 |
Codecov Report
@@ Coverage Diff @@
## main #1999 +/- ##
=======================================
Coverage 99.99% 99.99%
=======================================
Files 179 179
Lines 15738 15770 +32
=======================================
+ Hits 15737 15769 +32
Misses 1 1
📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
Summary: This commit improves the robustness of `fatmax` and `logsumexp` for inputs with infinities. - In constrast to `torch.logsumexp`, `logsumexp` does not give rise to `NaN`s in its backward pass even if infinities are present. - `fatmax` is updated to exhibit the same behavior in the presence of infinities, and now allows for the specification of an `alpha` parameter, which controls the the asymptotic power decay of the fat-tailed approximation. In addition, the commit introduces helper functions derivative of `logsumexp` and `fatmax`, e.g. `logplusexp`, `fatminimum`, `fatmaximum`, fixes a similar infinity issue with `logdiffexp`, and improves the associated test suite. Reviewed By: Balandat Differential Revision: D48878020
SebastianAment
force-pushed
the
export-D48878020
branch
from
fec94c5 to
c7b1f0a
Compare
|
This pull request was exported from Phabricator. Differential Revision: D48878020 |
|
This pull request has been merged in 9649b1c. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
This commit improves the robustness of
fatmaxandlogsumexpfor inputs with infinities.torch.logsumexp,logsumexpdoes not give rise toNaNs in its backward pass even if infinities are present.fatmaxis updated to exhibit the same behavior in the presence of infinities, and now allows for the specification of analphaparameter, which controls the the asymptotic power decay of the fat-tailed approximation.In addition, the commit introduces helper functions derivative of
logsumexpandfatmax, e.g.logplusexp,fatminimum,fatmaximum, fixes a similar infinity issue withlogdiffexp, and improves the associated test suite.Differential Revision: D48878020