Kernel hint design #6790
Merged
Kernel hint design #6790
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
reyoung
reviewed
Dec 20, 2017
doc/design/kernel_hint_design.md
Outdated
| const std::string kUseCUDNN = "use_cudnn"; | ||
| const std::string kUseMKLDNN = "use_mkldnn"; | ||
|
|
||
| KernelType GetExpectedKernelTyp() { |
reyoung
reviewed
Dec 20, 2017
doc/design/kernel_hint_design.md
Outdated
| LayoutType layout_; | ||
| }; | ||
| ``` | ||
| `place_` `data_type_` and `layout_` can come from the input tensor of the operator, `GetActualKernelType(inputs)` use inputs to infer the proper kernel key that fit the incoming data, user can not config it. |
There was a problem hiding this comment.
Users cannot configure it.
Actually, users can configure input data and influence the actual kernel type.
reyoung
reviewed
Dec 20, 2017
doc/design/kernel_hint_design.md
Outdated
| ## Solution | ||
|
|
||
| ### potential choice | ||
| 1, Do nothing, let the user add the information they want to operator‘s attribute and get them inside `GetExpectedKernelType`, this can work right. But there is a little problem that users may define many kinds of hints for the same purpose, such as `force_cpu`, `use_cpu`, `CPU` for CPU kernel, and `use_cudnn`, `force_cudnn`, `cudnn_kernel` for use of CUDNN kernel. |
reyoung
reviewed
Dec 20, 2017
doc/design/kernel_hint_design.md
Outdated
| ## Solution | ||
|
|
||
| ### potential choice | ||
| 1, Do nothing, let the user add the information they want to operator‘s attribute and get them inside `GetExpectedKernelType`, this can work right. But there is a little problem that users may define many kinds of hints for the same purpose, such as `force_cpu`, `use_cpu`, `CPU` for CPU kernel, and `use_cudnn`, `force_cudnn`, `cudnn_kernel` for use of CUDNN kernel. |
There was a problem hiding this comment.
We can define some global constant for these attribute name.
reyoung
reviewed
Dec 20, 2017
doc/design/kernel_hint_design.md
Outdated
|
|
||
| ### final choice | ||
| To provide enough flexibility while avoiding confusion definition, we can predefine some options, such as `force_cpu`, `use_cudnn`, `use_mkldnn` for a user to choose. | ||
|
|
There was a problem hiding this comment.
Using string to represent force_cpu, use_cudnn, use_mkldnn is not extendable. How to make an operator use CPU forcibly and use MKLDNN?
reyoung
reviewed
Dec 20, 2017
doc/design/kernel_hint_design.md
Outdated
| const std::string kUseMKLDNN = "use_mkldnn"; | ||
|
|
||
| KernelType GetExpectedKernelTyp() { | ||
| // "kernel_hint" is a user defined attribute name |
There was a problem hiding this comment.
I do not think kernel_hint is a formal name. It is confusing for users.
There was a problem hiding this comment.
done, use force_cpu as key and bool as value.
reyoung
reviewed
Dec 20, 2017
doc/design/kernel_hint_design.md
Outdated
|
|
||
| KernelType GetExpectedKernelTyp() { | ||
| // "kernel_hint" is a user defined attribute name | ||
| if (Attr<std::string>("kernel_hint") == kForceCPU) { |
There was a problem hiding this comment.
Why not use Attr<bool>(kForceCPU) ?
reyoung
previously approved these changes
Dec 20, 2017
doc/design/kernel_hint_design.md
Outdated
| const std::string kUseMKLDNN = "use_mkldnn"; | ||
|
|
||
| KernelType GetExpectedKernelType() { | ||
| if (Attr<bool>(kForceCPU)) { |
doc/design/kernel_hint_design.md
Outdated
| FORCE_CPU = core.kForceCPU() | ||
|
|
||
| def xx_layer(..., force_cpu=false): | ||
| layer_helper = LayerHelper(...) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.