Orthogonal Additive Kernels #1869
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
facebook-github-bot
added
CLA Signed
|
This pull request was exported from Phabricator. Differential Revision: D45217852 |
SebastianAment
added a commit
to SebastianAment/botorch
that referenced
this pull request
Jun 6, 2023
Summary: Pull Request resolved: pytorch#1869 OAKs were introduced in [Additive Gaussian Processes Revisited](https://arxiv.org/pdf/2206.09861.pdf) but were limited to Gaussian kernels with Gaussian data densities (which required the application of normalizing flows to the actual input data to make it look Gaussian). This commit introduces a generalization of OAKs that works with arbitrary base kernels by leveraging Gauss-Legendre quadrature rules for the associated one-dimensional integrals. OAKs could be more sample-efficient than canonical kernels in higher dimensions, and allow for more efficient relevance determination, because dimensions or interactions of dimensions can be pruned by setting their assoicated coefficients -- not just their lengthscales -- to zero. Reviewed By: Balandat Differential Revision: D45217852 fbshipit-source-id: 8af544145b0c3d0c280accc6d8192971f6f22a72
SebastianAment
force-pushed
the
export-D45217852
branch
from
9bbde2e to
1f54e6a
Compare
|
This pull request was exported from Phabricator. Differential Revision: D45217852 |
Codecov Report
@@ Coverage Diff @@
## main #1869 +/- ##
=========================================
Coverage 100.00% 100.00%
=========================================
Files 171 172 +1
Lines 15006 15093 +87
=========================================
+ Hits 15006 15093 +87
📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
SebastianAment
added a commit
to SebastianAment/botorch
that referenced
this pull request
Jun 6, 2023
Summary: Pull Request resolved: pytorch#1869 OAKs were introduced in [Additive Gaussian Processes Revisited](https://arxiv.org/pdf/2206.09861.pdf) but were limited to Gaussian kernels with Gaussian data densities (which required the application of normalizing flows to the actual input data to make it look Gaussian). This commit introduces a generalization of OAKs that works with arbitrary base kernels by leveraging Gauss-Legendre quadrature rules for the associated one-dimensional integrals. OAKs could be more sample-efficient than canonical kernels in higher dimensions, and allow for more efficient relevance determination, because dimensions or interactions of dimensions can be pruned by setting their assoicated coefficients -- not just their lengthscales -- to zero. Reviewed By: Balandat Differential Revision: D45217852 fbshipit-source-id: 3a124f7af1d9301bad84fc3fcc22f382b8f1693f
SebastianAment
force-pushed
the
export-D45217852
branch
from
1f54e6a to
acb0eec
Compare
|
This pull request was exported from Phabricator. Differential Revision: D45217852 |
SebastianAment
added a commit
to SebastianAment/botorch
that referenced
this pull request
Jun 6, 2023
Summary: Pull Request resolved: pytorch#1869 OAKs were introduced in [Additive Gaussian Processes Revisited](https://arxiv.org/pdf/2206.09861.pdf) but were limited to Gaussian kernels with Gaussian data densities (which required the application of normalizing flows to the actual input data to make it look Gaussian). This commit introduces a generalization of OAKs that works with arbitrary base kernels by leveraging Gauss-Legendre quadrature rules for the associated one-dimensional integrals. OAKs could be more sample-efficient than canonical kernels in higher dimensions, and allow for more efficient relevance determination, because dimensions or interactions of dimensions can be pruned by setting their assoicated coefficients -- not just their lengthscales -- to zero. Reviewed By: Balandat Differential Revision: D45217852 fbshipit-source-id: 2ed6911e0cdf4da526690df98bc9f392712e29bc
SebastianAment
force-pushed
the
export-D45217852
branch
from
acb0eec to
4592593
Compare
|
This pull request was exported from Phabricator. Differential Revision: D45217852 |
SebastianAment
added a commit
to SebastianAment/botorch
that referenced
this pull request
Jun 6, 2023
Summary: Pull Request resolved: pytorch#1869 OAKs were introduced in [Additive Gaussian Processes Revisited](https://arxiv.org/pdf/2206.09861.pdf) but were limited to Gaussian kernels with Gaussian data densities (which required the application of normalizing flows to the actual input data to make it look Gaussian). This commit introduces a generalization of OAKs that works with arbitrary base kernels by leveraging Gauss-Legendre quadrature rules for the associated one-dimensional integrals. OAKs could be more sample-efficient than canonical kernels in higher dimensions, and allow for more efficient relevance determination, because dimensions or interactions of dimensions can be pruned by setting their assoicated coefficients -- not just their lengthscales -- to zero. Reviewed By: Balandat Differential Revision: D45217852 fbshipit-source-id: 10059298add80ba49ad48d7a0484a8cc2fb12f32
SebastianAment
force-pushed
the
export-D45217852
branch
from
4592593 to
009ae77
Compare
|
This pull request was exported from Phabricator. Differential Revision: D45217852 |
SebastianAment
added a commit
to SebastianAment/botorch
that referenced
this pull request
Jun 7, 2023
Summary: Pull Request resolved: pytorch#1869 OAKs were introduced in [Additive Gaussian Processes Revisited](https://arxiv.org/pdf/2206.09861.pdf) but were limited to Gaussian kernels with Gaussian data densities (which required the application of normalizing flows to the actual input data to make it look Gaussian). This commit introduces a generalization of OAKs that works with arbitrary base kernels by leveraging Gauss-Legendre quadrature rules for the associated one-dimensional integrals. OAKs could be more sample-efficient than canonical kernels in higher dimensions, and allow for more efficient relevance determination, because dimensions or interactions of dimensions can be pruned by setting their assoicated coefficients -- not just their lengthscales -- to zero. Reviewed By: Balandat Differential Revision: D45217852 fbshipit-source-id: 7024d5fce99d943c16cc936d007dc497224d2912
SebastianAment
force-pushed
the
export-D45217852
branch
from
009ae77 to
e1002ca
Compare
|
This pull request was exported from Phabricator. Differential Revision: D45217852 |
SebastianAment
added a commit
to SebastianAment/botorch
that referenced
this pull request
Jun 7, 2023
Summary: Pull Request resolved: pytorch#1869 OAKs were introduced in [Additive Gaussian Processes Revisited](https://arxiv.org/pdf/2206.09861.pdf) but were limited to Gaussian kernels with Gaussian data densities (which required the application of normalizing flows to the actual input data to make it look Gaussian). This commit introduces a generalization of OAKs that works with arbitrary base kernels by leveraging Gauss-Legendre quadrature rules for the associated one-dimensional integrals. OAKs could be more sample-efficient than canonical kernels in higher dimensions, and allow for more efficient relevance determination, because dimensions or interactions of dimensions can be pruned by setting their assoicated coefficients -- not just their lengthscales -- to zero. Reviewed By: Balandat Differential Revision: D45217852 fbshipit-source-id: 650c42daf0b8e5eab3735913b723cb8ea3bacacf
SebastianAment
force-pushed
the
export-D45217852
branch
from
e1002ca to
7a8222d
Compare
|
This pull request was exported from Phabricator. Differential Revision: D45217852 |
1 similar comment
|
This pull request was exported from Phabricator. Differential Revision: D45217852 |
SebastianAment
added a commit
to SebastianAment/botorch
that referenced
this pull request
Jun 7, 2023
Summary: Pull Request resolved: pytorch#1869 OAKs were introduced in [Additive Gaussian Processes Revisited](https://arxiv.org/pdf/2206.09861.pdf) but were limited to Gaussian kernels with Gaussian data densities (which required the application of normalizing flows to the actual input data to make it look Gaussian). This commit introduces a generalization of OAKs that works with arbitrary base kernels by leveraging Gauss-Legendre quadrature rules for the associated one-dimensional integrals. OAKs could be more sample-efficient than canonical kernels in higher dimensions, and allow for more efficient relevance determination, because dimensions or interactions of dimensions can be pruned by setting their assoicated coefficients -- not just their lengthscales -- to zero. Reviewed By: Balandat Differential Revision: D45217852 fbshipit-source-id: c2ef5d82e3dff5c7a2ce34e678686a29cc236351
SebastianAment
force-pushed
the
export-D45217852
branch
from
7a8222d to
6751493
Compare
|
This pull request was exported from Phabricator. Differential Revision: D45217852 |
SebastianAment
added a commit
to SebastianAment/botorch
that referenced
this pull request
Jun 7, 2023
Summary: Pull Request resolved: pytorch#1869 OAKs were introduced in [Additive Gaussian Processes Revisited](https://arxiv.org/pdf/2206.09861.pdf) but were limited to Gaussian kernels with Gaussian data densities (which required the application of normalizing flows to the actual input data to make it look Gaussian). This commit introduces a generalization of OAKs that works with arbitrary base kernels by leveraging Gauss-Legendre quadrature rules for the associated one-dimensional integrals. OAKs could be more sample-efficient than canonical kernels in higher dimensions, and allow for more efficient relevance determination, because dimensions or interactions of dimensions can be pruned by setting their assoicated coefficients -- not just their lengthscales -- to zero. Reviewed By: Balandat Differential Revision: D45217852 fbshipit-source-id: 1048ce03f6afcb5bc4df6f2e4fa8032b217a1b23
SebastianAment
force-pushed
the
export-D45217852
branch
from
6751493 to
d711be2
Compare
Summary: Pull Request resolved: pytorch#1869 OAKs were introduced in [Additive Gaussian Processes Revisited](https://arxiv.org/pdf/2206.09861.pdf) but were limited to Gaussian kernels with Gaussian data densities (which required the application of normalizing flows to the actual input data to make it look Gaussian). This commit introduces a generalization of OAKs that works with arbitrary base kernels by leveraging Gauss-Legendre quadrature rules for the associated one-dimensional integrals. OAKs could be more sample-efficient than canonical kernels in higher dimensions, and allow for more efficient relevance determination, because dimensions or interactions of dimensions can be pruned by setting their assoicated coefficients -- not just their lengthscales -- to zero. Reviewed By: Balandat Differential Revision: D45217852 fbshipit-source-id: 37efabffbfad8ca32c211e80a468bf2decad140e
SebastianAment
force-pushed
the
export-D45217852
branch
from
d711be2 to
c75ac05
Compare
|
This pull request was exported from Phabricator. Differential Revision: D45217852 |
|
This pull request has been merged in 6d329a8. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
OAKs were introduced in Additive Gaussian Processes Revisited but were limited to Gaussian kernels with Gaussian data densities (which required the application of normalizing flows to the actual input data to make it look Gaussian).
This commit introduces a generalization of OAKs that works with arbitrary base kernels by leveraging Gauss-Legendre quadrature rules for the associated one-dimensional integrals.
OAKs could be more sample-efficient than canonical kernels in higher dimensions, and allow for more efficient relevance determination, because dimensions or interactions of dimensions can be pruned by setting their assoicated coefficients -- not just their lengthscales -- to zero.
Reviewed By: Balandat
Differential Revision: D45217852