Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ONNX] QLinearConv Support #8007

Merged
merged 4 commits into from
May 13, 2021
Merged

[ONNX] QLinearConv Support #8007

merged 4 commits into from
May 13, 2021

Conversation

huochaitiantang
Copy link
Contributor

Thanks for contributing to TVM! Please refer to guideline https://tvm.apache.org/docs/contribute/ for useful information and tips. After the pull request is submitted, please request code reviews from Reviewers by @ them in the pull request thread.

Copy link
Contributor

@mbrookhart mbrookhart left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like it needs a rebase? I'm not sure I understand the issue with the onnx tests, is that something we can work around?

Comment on lines 4231 to 4232
# For QLinearConv tests in ONNX, scale and zero_point input are placeholders,
# but qnn.conv2d requires scale and zero_point input to be scalar
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you explain this comment more?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have tried to work around this issue but failed. For ONNX test model of QLinearConv, y_scale is input node and will be parsed to free_var in relay. It will cause an error while lowering requantize because it requires output_scale (y_scale) to be constant expr:

E     1: tvm::relay::qnn::RequantizeLower(tvm::RelayExpr const&, tvm::RelayExpr const&, tvm::RelayExpr const&, tvm::RelayExpr const&, tvm::RelayExpr const&, tvm::relay::qnn::RequantizeAttrs const*, tvm::r
untime::Array<tvm::PrimExpr, void> const&, tvm::runtime::DataType const&)
E           at /home/liuliang/Desktop/tvms/tvm/src/relay/qnn/op/requantize.cc:148
E     0: float tvm::relay::GetScalarFromConstant<float>(tvm::RelayExpr)
E           at /home/liuliang/Desktop/tvms/tvm/src/relay/qnn/op/../../transforms/pattern_utils.h:514
E     File "/home/liuliang/Desktop/tvms/tvm/src/relay/quantize/../transforms/pattern_utils.h", line 514
E   TVMError: 
E   ---------------------------------------------------------------
E   An error occurred during the execution of TVM.
E   For more information, please see: https://tvm.apache.org/docs/errors.html
E   ---------------------------------------------------------------
E     Check failed: (n) is false: Expr must be a constant expr - #[version = "0.0.5"]
E   free_var %y_scale: float32;
E   cast(%y_scale, dtype="float32") /* ty=float32 */

Do you have some suggestions to solve it?

@mbrookhart
Copy link
Contributor

Hmm, what do you think about checking to see if the scale is constant, and if it's not, doing dequantize -> quantize instead of requantize?

@huochaitiantang
Copy link
Contributor Author

@mbrookhart, thanks! The idea looks good, it could solve the issue!

Copy link
Contributor

@mbrookhart mbrookhart left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Looks like we had a CI outage, can you rebase to kick off a new job?

@mbrookhart mbrookhart merged commit 158aedd into apache:main May 13, 2021
@mbrookhart
Copy link
Contributor

Thanks @huochaitiantang !

mehrdadh pushed a commit to mehrdadh/tvm that referenced this pull request May 19, 2021
* Add QLinearConv for onnx frontend

* Reformat

* Squeeze 1D tensor for weight_scale & weight_zero_point

* Doing dequatize -> quantize if y_scale is not constant
trevor-m pushed a commit to trevor-m/tvm that referenced this pull request Jun 17, 2021
* Add QLinearConv for onnx frontend

* Reformat

* Squeeze 1D tensor for weight_scale & weight_zero_point

* Doing dequatize -> quantize if y_scale is not constant
trevor-m pushed a commit to neo-ai/tvm that referenced this pull request Jun 17, 2021
* Add QLinearConv for onnx frontend

* Reformat

* Squeeze 1D tensor for weight_scale & weight_zero_point

* Doing dequatize -> quantize if y_scale is not constant
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants