-
Notifications
You must be signed in to change notification settings - Fork 79
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement multi-dimensional reduction and refactor cuTENSOR support #2430
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please add the include suggestion, but then !
|
||
#include <cuda_runtime.h> | ||
#include <cutensor.h> | ||
|
||
/** | ||
* The interface below is designed for CUTENSOR v1. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah... should we update to v2? Separate PR obviously.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We might want to, it’s more capable but we’re not using those capabilities
include/lbann/utils/CMakeLists.txt
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't know if this is the right place for these utilities. The RowMajor
/ColMajor
stuff is largely setup for the cuTENSOR/cuTT stuff. I might prefer src/layers/{helpers,common,utils}
or something like any of those? On the other hand, there's a lot of stuff in this directory that's probably a bit misfiled, so it's not unreasonable to leave it here.
Co-authored-by: Tom Benson <benson31@llnl.gov>
@bvanessen tests are passing |
This PR adds a new layer that performs reductions on specific tensor dimensions. For training (i.e., backpropagation), only the "sum" operation is supported.
Fixes Issue #2429.