Destination flipping for add!!(x, y)
#346
Labels
inplace accumulation
for things relating to inplace accumulation of gradients
add!!(x, y)
#346
Right now
add!!(x, y)
will only ever mutatex
, nevery
.because of the logically linear nature of AD,
y
being a differential (i.e. the output of pullback) means it is used exactly once .Which means we are infact free to mutate it, also since this will be it's only use. (same also says why we are always ok to mutate
x
)See google-research/dex-lang#401 (comment)
(and to a lesser extent https://twitter.com/oxinabox_frames/status/1387722380978622468)
So we could pick either
x
ory
depending on which was a validinplacable_destination
.And maybe also always chose
Array
if one of them is anArray
(since that is the best destination if that has been allocated)This would solve at least the basic cases of #345
The text was updated successfully, but these errors were encountered: