Add support of custom _forw functions
#1037
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
| Expr* customForwPassCE = | ||
| m_Builder.BuildCallToCustomDerivativeOrNumericalDiff( | ||
| forwPassFnName, args, getCurrentScope(), | ||
| const_cast<DeclContext*>(FD->getDeclContext())); |
There was a problem hiding this comment.
warning: do not use const_cast [cppcoreguidelines-pro-type-const-cast]
const_cast<DeclContext*>(FD->getDeclContext()));
^
infinite-void-16
force-pushed
the
CustomForw
branch
from
5d64c63 to
08d8eba
Compare
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #1037 +/- ##
==========================================
+ Coverage 94.09% 94.12% +0.02%
==========================================
Files 55 55
Lines 8250 8275 +25
==========================================
+ Hits 7763 7789 +26
+ Misses 487 486 -1
... and 2 files with indirect coverage changes
|
infinite-void-16
force-pushed
the
CustomForw
branch
2 times, most recently
from
58129cd to
a14f6aa
Compare
| m_Builder.BuildCallToCustomDerivativeOrNumericalDiff( | ||
| forwPassFnName, args, getCurrentScope(), | ||
| const_cast<DeclContext*>(FD->getDeclContext())); | ||
| return customForwPassCE; |
There was a problem hiding this comment.
warning: do not use const_cast [cppcoreguidelines-pro-type-const-cast]
const_cast<DeclContext*>(FD->getDeclContext()));
^
infinite-void-16
force-pushed
the
CustomForw
branch
from
a14f6aa to
bbc015b
Compare
infinite-void-16
force-pushed
the
CustomForw
branch
from
bbc015b to
d77cb5e
Compare
infinite-void-16
changed the title
_forw functions
vgvassilev
approved these changes
Aug 11, 2024
There was a problem hiding this comment.
Lgtm! @PetroZarytskyi, can you take a look at the change in the TBR part?
infinite-void-16
force-pushed
the
CustomForw
branch
from
d77cb5e to
bcf7911
Compare
vgvassilev
reviewed
Aug 19, 2024
This commit adds support for custom (user-provided) `_forw` functions.
A `_forw` function, if available, is called in place of the actual
function.
For example, if the primal code contains:
```cpp
someFn(u, v, w);
```
and user has defined a custom `_reverse_forw` function for `someFn` as follows:
```cpp
namespace clad {
namespace custom_derivatives {
void someFn_reverse_forw(double u, double v, double w, double *d_u,
double *d_v, double *dw) {
// ...
// ...
}
}
}
```
Then clad will generate the derivative function as follows:
```cpp
// forward-pass
clad::custom_derivatives::someFn_reverse_forw(u, v, w, d_u, d_v, d_w);
// ...
// reverse-pass; no change in reverse-pass
someFn_pullback(u, v, w, d_u, d_v, d_w);
// ...
```
But more importantly, why do we need such a functionality? Two reasons:
- Supporting reference/pointer return types in the reverse-mode. This
has been discussed at great length here:
vgvassilev#425 (vgvassilev#425)
- Supporting types whose elements grows dynamically, such as
`std::vector` and `std::map`. The issue is that we correctly
need to update the size/property of the adjoint variable when a
function call updates the size/property of the corresponding primal
variable. For example: a call to `vec.push_back(...)` should update
the size of `_d_vec` as well. However, the actual function call does
not modify the adjoint variable in any way. Here comes `_forw` functions
to the rescue. `_forw` functions makes it possible to adjust the adjoint
variable size/properties along with executing the actual function call.
Please note that `_reverse_forw` function signature takes adjoint variables as
arguments and return `clad::ValueAndAdjoint<U, V>` to support the
reference/pointer return type.
infinite-void-16
force-pushed
the
CustomForw
branch
from
bcf7911 to
6ede83c
Compare
This was referenced Aug 22, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This commit adds support for custom (user-provided)
_forwfunctions.A
_forwfunction, if available, is called in place of the actualfunction.
For example, if the primal code contains:
someFn(u, v, w);and user has defined a custom
_forwfunction forsomeFnas follows:Then clad will generate the derivative function as follows:
But more importantly, why do we need such a functionality? Two reasons:
Supporting reference/pointer return types in the reverse-mode. This
has been discussed at great length here:
Add initial support for diff of ref return types in rev mode #425 (Add initial support for diff of ref return types in rev mode #425)
Supporting types whose elements grows dynamically, such as
std::vectorandstd::map. The issue is that we correctlyneed to update the size/property of the adjoint variable when a
function call updates the size/property of the corresponding primal
variable. For example: a call to
vec.push_back(...)should updatethe size of
_d_vecas well. However, the actual function call doesnot modify the adjoint variable in any way. Here comes
_forwfunctionsto the rescue.
_forwfunctions makes it possible to adjust the adjointvariable size/properties along with executing the actual function call.
Please note that
_forwfunction signature takes adjoint variables asarguments and return
clad::ValueAndAdjoint<U, V>to support thereference/pointer return type.