-
Notifications
You must be signed in to change notification settings - Fork 6.8k
Update CustomOp doc with changes for GPU support #17486
Conversation
@rondogency please add the fix for the gpu example test program here from #17270 (comment) |
@samskalicky the fix is in #17516 and you can review it now |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Any idea why the formatting changes to italics here?
https://github.com/apache/incubator-mxnet/pull/17486/files/1ef0af458f8496f7a7f8d0e6d6300904afeb2b56#diff-1727b2876a104444973921644eac97bdR255
@aaronmarkham it looks fine to me here: Looks like its just weird GitHub source-diff showing odd italics. |
Hi, @rondogency I am trying to use the dynamic operator and have some questions. |
Hi @a550461053, MXNet custom operators have been designed not to integrate with any MXNet backend classes (mxnet::NDArray) to simplify building. We've designed MXTensor to be equivalent. Is there some other functionality you were wanting from NDArray thats not in MXTensor? The mutateInputs API allows you to do the same as the backend APIs, so you can specify the index of the input that you want to be mutable to make a BatchNorm-like operator. |
Ok, thank you. I want to create a custom operator calling another operator which input NDArray. Both operator is async pushed to engine, I think this way is possible. Also, I am not clear on using mutateInputs API, how and when to use it? If you can provide a example of the mutateInputs API, I will be grateful to you~ |
Hi @a550461053 currently the custom operator design focused on separating the custom operator from the MXNet backend source code complexity. This means that your custom operator can (must) be entirely separated from MXNet. So you cannot call a regular built-in MXNet operator from your custom operator. We have an item here #17006 for adding support in the future to be able to do this, but it is not implemented yet. As for how to use mutateInputs, it works exactly as the doc describes:
So lets say you have an operator with 5 inputs, you can mark the indices of the inputs that you want to be mutable like this (for example mark the last two inputs as mutable):
|
@a550461053 Thanks for the questions. Currently you cannot call MXNet built-in operators from a custom operator. We are making a plan for adding this support, but it cannot be easily implemented and may be a separate project. Also I added a snippet of how to use mutable input, hope it would help. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, thanks @rondogency!
Thank you and @rondogency . I am clear on use of mutateInputs API. |
@a550461053 just to clarify, you do not need to recompile MXNet from source to use a custom operator, you can dynamically load your library into MXNet at runtime using one of the nightly builds from here: https://repo.mxnet.io/dist/index.html |
@aaronmarkham or @wkcn can we merge this please? |
The PR has been merged. Thanks for your contribution! : ) |
* add gpu doc portion * resolve sam comments * add lib_api and resolve comments * resolve sam comments * try remove italic * fix typo and add mutable example * retrigger ci
* add gpu doc portion * resolve sam comments * add lib_api and resolve comments * resolve sam comments * try remove italic * fix typo and add mutable example * retrigger ci
Description
Improve the custom operator example doc by adding how to write GPU custom operator.
This is a continuation of the CustomOp tutorial doc #17241
This PR is not
Those will be addressed in the coming PRs
Checklist
Essentials
Please feel free to remove inapplicable items for your PR.
Changes
Comments