run opted out decompositions #3646
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
bdhirsh
force-pushed
the
functionalization_decompositions
branch
from
83d30ed to
8c7e2ec
Compare
|
cc @JackCaoG this is the "there are a handful of problematic decompositions" thing I mentioned a few weeks ago |
bdhirsh
force-pushed
the
functionalization_decompositions
branch
from
8c7e2ec to
3ceb49c
Compare
bdhirsh
added a commit
to pytorch/pytorch
that referenced
this pull request
Jun 13, 2022
…p; other functionalization fixes" I moved out the changes to `FunctionalTensorWrapper.h` from the LTC <> functionalization PR into a separate PR here, so dealing with XLA failures will be a bit easier. Specifically, the LTC PR will make a few operators like `pixel_shuffle` that are functional, but decompose into view ops, require re-functionalization once they hit the XLA backend. This PR exposes a helper utility to do that through `functionalize_aten_op`. This PR also contains the changes to: - fix `detach()` for `FunctionalTensorWrapper` - fix some undefined tensor handling cases I have an XLA patch here to do the re-functionalizing: pytorch/xla#3646 [ghstack-poisoned]
bdhirsh
added a commit
to pytorch/pytorch
that referenced
this pull request
Jun 13, 2022
…nalization fixes" I moved out the changes to `FunctionalTensorWrapper.h` from the LTC <> functionalization PR into a separate PR here, so dealing with XLA failures will be a bit easier. Specifically, the LTC PR will make a few operators like `pixel_shuffle` that are functional, but decompose into view ops, require re-functionalization once they hit the XLA backend. This PR exposes a helper utility to do that through `functionalize_aten_op`. This PR also contains the changes to: - fix `detach()` for `FunctionalTensorWrapper` - fix some undefined tensor handling cases I have an XLA patch here to do the re-functionalizing: pytorch/xla#3646 [ghstack-poisoned]
bdhirsh
added a commit
to pytorch/pytorch
that referenced
this pull request
Jun 13, 2022
…p; other functionalization fixes" I moved out the changes to `FunctionalTensorWrapper.h` from the LTC <> functionalization PR into a separate PR here, so dealing with XLA failures will be a bit easier. Specifically, the LTC PR will make a few operators like `pixel_shuffle` that are functional, but decompose into view ops, require re-functionalization once they hit the XLA backend. This PR exposes a helper utility to do that through `functionalize_aten_op`. This PR also contains the changes to: - fix `detach()` for `FunctionalTensorWrapper` - fix some undefined tensor handling cases I have an XLA patch here to do the re-functionalizing: pytorch/xla#3646 [ghstack-poisoned]
bdhirsh
added a commit
to pytorch/pytorch
that referenced
this pull request
Jun 13, 2022
…nalization fixes" I moved out the changes to `FunctionalTensorWrapper.h` from the LTC <> functionalization PR into a separate PR here, so dealing with XLA failures will be a bit easier. Specifically, the LTC PR will make a few operators like `pixel_shuffle` that are functional, but decompose into view ops, require re-functionalization once they hit the XLA backend. This PR exposes a helper utility to do that through `functionalize_aten_op`. This PR also contains the changes to: - fix `detach()` for `FunctionalTensorWrapper` - fix some undefined tensor handling cases I have an XLA patch here to do the re-functionalizing: pytorch/xla#3646 [ghstack-poisoned]
bdhirsh
force-pushed
the
functionalization_decompositions
branch
from
3ceb49c to
939566d
Compare
bdhirsh
added a commit
to pytorch/pytorch
that referenced
this pull request
Jun 14, 2022
…p; other functionalization fixes" I moved out the changes to `FunctionalTensorWrapper.h` from the LTC <> functionalization PR into a separate PR here, so dealing with XLA failures will be a bit easier. Specifically, the LTC PR will make a few operators like `pixel_shuffle` that are functional, but decompose into view ops, require re-functionalization once they hit the XLA backend. This PR exposes a helper utility to do that through `functionalize_aten_op`. This PR also contains the changes to: - fix `detach()` for `FunctionalTensorWrapper` - fix some undefined tensor handling cases I have an XLA patch here to do the re-functionalizing: pytorch/xla#3646 [ghstack-poisoned]
bdhirsh
added a commit
to pytorch/pytorch
that referenced
this pull request
Jun 14, 2022
…nalization fixes" I moved out the changes to `FunctionalTensorWrapper.h` from the LTC <> functionalization PR into a separate PR here, so dealing with XLA failures will be a bit easier. Specifically, the LTC PR will make a few operators like `pixel_shuffle` that are functional, but decompose into view ops, require re-functionalization once they hit the XLA backend. This PR exposes a helper utility to do that through `functionalize_aten_op`. This PR also contains the changes to: - fix `detach()` for `FunctionalTensorWrapper` - fix some undefined tensor handling cases I have an XLA patch here to do the re-functionalizing: pytorch/xla#3646 [ghstack-poisoned]
bdhirsh
added a commit
to pytorch/pytorch
that referenced
this pull request
Jun 15, 2022
…p; other functionalization fixes" I moved out the changes to `FunctionalTensorWrapper.h` from the LTC <> functionalization PR into a separate PR here, so dealing with XLA failures will be a bit easier. Specifically, the LTC PR will make a few operators like `pixel_shuffle` that are functional, but decompose into view ops, require re-functionalization once they hit the XLA backend. This PR exposes a helper utility to do that through `functionalize_aten_op`. This PR also contains the changes to: - fix `detach()` for `FunctionalTensorWrapper` - fix some undefined tensor handling cases I have an XLA patch here to do the re-functionalizing: pytorch/xla#3646 [ghstack-poisoned]
bdhirsh
added a commit
to pytorch/pytorch
that referenced
this pull request
Jun 15, 2022
…nalization fixes" I moved out the changes to `FunctionalTensorWrapper.h` from the LTC <> functionalization PR into a separate PR here, so dealing with XLA failures will be a bit easier. Specifically, the LTC PR will make a few operators like `pixel_shuffle` that are functional, but decompose into view ops, require re-functionalization once they hit the XLA backend. This PR exposes a helper utility to do that through `functionalize_aten_op`. This PR also contains the changes to: - fix `detach()` for `FunctionalTensorWrapper` - fix some undefined tensor handling cases I have an XLA patch here to do the re-functionalizing: pytorch/xla#3646 [ghstack-poisoned]
bdhirsh
added a commit
to pytorch/pytorch
that referenced
this pull request
Jun 15, 2022
…p; other functionalization fixes" I moved out the changes to `FunctionalTensorWrapper.h` from the LTC <> functionalization PR into a separate PR here, so dealing with XLA failures will be a bit easier. Specifically, the LTC PR will make a few operators like `pixel_shuffle` that are functional, but decompose into view ops, require re-functionalization once they hit the XLA backend. This PR exposes a helper utility to do that through `functionalize_aten_op`. This PR also contains the changes to: - fix `detach()` for `FunctionalTensorWrapper` - fix some undefined tensor handling cases I have an XLA patch here to do the re-functionalizing: pytorch/xla#3646 [ghstack-poisoned]
bdhirsh
added a commit
to pytorch/pytorch
that referenced
this pull request
Jun 15, 2022
…nalization fixes" I moved out the changes to `FunctionalTensorWrapper.h` from the LTC <> functionalization PR into a separate PR here, so dealing with XLA failures will be a bit easier. Specifically, the LTC PR will make a few operators like `pixel_shuffle` that are functional, but decompose into view ops, require re-functionalization once they hit the XLA backend. This PR exposes a helper utility to do that through `functionalize_aten_op`. This PR also contains the changes to: - fix `detach()` for `FunctionalTensorWrapper` - fix some undefined tensor handling cases I have an XLA patch here to do the re-functionalizing: pytorch/xla#3646 [ghstack-poisoned]
bdhirsh
force-pushed
the
functionalization_decompositions
branch
3 times, most recently
from
552fab9 to
29af626
Compare
|
I realized that we can't actually have XLA use the "functionalize under the hood" helper yet, because pt/xla doesn't have (1) this PR: manually register the ~9-10 decompositions from core, by just calling into the decompositions directly (e.g. (2) I'll land my TS integration PR, (which would have broken xla if not for landing this pR first) (3) pt/xla opts into the functionalization pass at some point later, which'll require updating the decomps in this PR to use the |
bdhirsh
force-pushed
the
functionalization_decompositions
branch
from
29af626 to
c673d9e
Compare
|
@pytorchbot merge |
bdhirsh
added a commit
to pytorch/pytorch
that referenced
this pull request
Jun 16, 2022
…p; other functionalization fixes" I moved out the changes to `FunctionalTensorWrapper.h` from the LTC <> functionalization PR into a separate PR here, so dealing with XLA failures will be a bit easier. Specifically, the LTC PR will make a few operators like `pixel_shuffle` that are functional, but decompose into view ops, require re-functionalization once they hit the XLA backend. This PR exposes a helper utility to do that through `functionalize_aten_op`. This PR also contains the changes to: - fix `detach()` for `FunctionalTensorWrapper` - fix some undefined tensor handling cases I have an XLA patch here to do the re-functionalizing: pytorch/xla#3646 [ghstack-poisoned]
bdhirsh
added a commit
to pytorch/pytorch
that referenced
this pull request
Jun 16, 2022
…nalization fixes" I moved out the changes to `FunctionalTensorWrapper.h` from the LTC <> functionalization PR into a separate PR here, so dealing with XLA failures will be a bit easier. Specifically, the LTC PR will make a few operators like `pixel_shuffle` that are functional, but decompose into view ops, require re-functionalization once they hit the XLA backend. This PR exposes a helper utility to do that through `functionalize_aten_op`. This PR also contains the changes to: - fix `detach()` for `FunctionalTensorWrapper` - fix some undefined tensor handling cases I have an XLA patch here to do the re-functionalizing: pytorch/xla#3646 [ghstack-poisoned]
|
@bdhirsh Does pytorch bot works for pytorch/xla too? |
bdhirsh
added a commit
to pytorch/pytorch
that referenced
this pull request
Jun 17, 2022
…p; other functionalization fixes" I moved out the changes to `FunctionalTensorWrapper.h` from the LTC <> functionalization PR into a separate PR here, so dealing with XLA failures will be a bit easier. Specifically, the LTC PR will make a few operators like `pixel_shuffle` that are functional, but decompose into view ops, require re-functionalization once they hit the XLA backend. This PR exposes a helper utility to do that through `functionalize_aten_op`. This PR also contains the changes to: - fix `detach()` for `FunctionalTensorWrapper` - fix some undefined tensor handling cases I have an XLA patch here to do the re-functionalizing: pytorch/xla#3646 [ghstack-poisoned]
bdhirsh
added a commit
to pytorch/pytorch
that referenced
this pull request
Jun 17, 2022
…nalization fixes" I moved out the changes to `FunctionalTensorWrapper.h` from the LTC <> functionalization PR into a separate PR here, so dealing with XLA failures will be a bit easier. Specifically, the LTC PR will make a few operators like `pixel_shuffle` that are functional, but decompose into view ops, require re-functionalization once they hit the XLA backend. This PR exposes a helper utility to do that through `functionalize_aten_op`. This PR also contains the changes to: - fix `detach()` for `FunctionalTensorWrapper` - fix some undefined tensor handling cases I have an XLA patch here to do the re-functionalizing: pytorch/xla#3646 [ghstack-poisoned]
bdhirsh
added a commit
to pytorch/pytorch
that referenced
this pull request
Jun 17, 2022
…p; other functionalization fixes" I moved out the changes to `FunctionalTensorWrapper.h` from the LTC <> functionalization PR into a separate PR here, so dealing with XLA failures will be a bit easier. Specifically, the LTC PR will make a few operators like `pixel_shuffle` that are functional, but decompose into view ops, require re-functionalization once they hit the XLA backend. This PR exposes a helper utility to do that through `functionalize_aten_op`. This PR also contains the changes to: - fix `detach()` for `FunctionalTensorWrapper` - fix some undefined tensor handling cases I have an XLA patch here to do the re-functionalizing: pytorch/xla#3646 [ghstack-poisoned]
bdhirsh
added a commit
to pytorch/pytorch
that referenced
this pull request
Jun 17, 2022
…nalization fixes" I moved out the changes to `FunctionalTensorWrapper.h` from the LTC <> functionalization PR into a separate PR here, so dealing with XLA failures will be a bit easier. Specifically, the LTC PR will make a few operators like `pixel_shuffle` that are functional, but decompose into view ops, require re-functionalization once they hit the XLA backend. This PR exposes a helper utility to do that through `functionalize_aten_op`. This PR also contains the changes to: - fix `detach()` for `FunctionalTensorWrapper` - fix some undefined tensor handling cases I have an XLA patch here to do the re-functionalizing: pytorch/xla#3646 [ghstack-poisoned]
bdhirsh
added a commit
to pytorch/pytorch
that referenced
this pull request
Jun 21, 2022
…p; other functionalization fixes" I moved out the changes to `FunctionalTensorWrapper.h` from the LTC <> functionalization PR into a separate PR here, so dealing with XLA failures will be a bit easier. Specifically, the LTC PR will make a few operators like `pixel_shuffle` that are functional, but decompose into view ops, require re-functionalization once they hit the XLA backend. This PR exposes a helper utility to do that through `functionalize_aten_op`. This PR also contains the changes to: - fix `detach()` for `FunctionalTensorWrapper` - fix some undefined tensor handling cases I have an XLA patch here to do the re-functionalizing: pytorch/xla#3646 [ghstack-poisoned]
bdhirsh
added a commit
to pytorch/pytorch
that referenced
this pull request
Jun 21, 2022
…nalization fixes" I moved out the changes to `FunctionalTensorWrapper.h` from the LTC <> functionalization PR into a separate PR here, so dealing with XLA failures will be a bit easier. Specifically, the LTC PR will make a few operators like `pixel_shuffle` that are functional, but decompose into view ops, require re-functionalization once they hit the XLA backend. This PR exposes a helper utility to do that through `functionalize_aten_op`. This PR also contains the changes to: - fix `detach()` for `FunctionalTensorWrapper` - fix some undefined tensor handling cases I have an XLA patch here to do the re-functionalizing: pytorch/xla#3646 [ghstack-poisoned]
pytorchmergebot
added a commit
to pytorch/pytorch
that referenced
this pull request
Jun 22, 2022
…p; other functionalization fixes" I moved out the changes to `FunctionalTensorWrapper.h` from the LTC <> functionalization PR into a separate PR here, so dealing with XLA failures will be a bit easier. Specifically, the LTC PR will make a few operators like `pixel_shuffle` that are functional, but decompose into view ops, require re-functionalization once they hit the XLA backend. This PR exposes a helper utility to do that through `functionalize_aten_op`. This PR also contains the changes to: - fix `detach()` for `FunctionalTensorWrapper` - fix some undefined tensor handling cases I have an XLA patch here to do the re-functionalizing: pytorch/xla#3646 [ghstack-poisoned]
pytorchmergebot
added a commit
to pytorch/pytorch
that referenced
this pull request
Jun 22, 2022
…nalization fixes" I moved out the changes to `FunctionalTensorWrapper.h` from the LTC <> functionalization PR into a separate PR here, so dealing with XLA failures will be a bit easier. Specifically, the LTC PR will make a few operators like `pixel_shuffle` that are functional, but decompose into view ops, require re-functionalization once they hit the XLA backend. This PR exposes a helper utility to do that through `functionalize_aten_op`. This PR also contains the changes to: - fix `detach()` for `FunctionalTensorWrapper` - fix some undefined tensor handling cases I have an XLA patch here to do the re-functionalizing: pytorch/xla#3646 [ghstack-poisoned]
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
with functionalization, there are a handful of "problematic" decompositions in core, where we have a functional operator that decomposes into view operators after the functionalization pass has already run.
I've enumerated them here, and added a helper function so you can run the decomposition by "re-functionalizing" it.
Waiting to land this PR until after I've landed pytorch/pytorch#79420