Add qnn batch_matmul operator #8401
Conversation
elvin-n
commented
Jul 4, 2021
elvin-n
commented
- added support of the different out type for x86 batch_matmul
- add support of the different out type for x86 batch_matmul
There was a problem hiding this comment.
These changes look excellent, I really like how you've minimized code duplication in the shape functions. I'm not quite sure why CI is failing, the error message is saying that Relay expects the output shape to be [16, 32] instead of [16, 16] like its supposed to be. I couldn't find any change in the shape functions that would make relay think that but there must be something subtle.
|
|
||
| Parameters | ||
| ---------- | ||
| x : tvm.relay.Expr |
| if yzero_point_zero == True: | ||
| y_zero_point = 0 | ||
| else: | ||
| y_zero_point = -1 |
There was a problem hiding this comment.
Should test on more non-zero zero points.
There was a problem hiding this comment.
@masahi just curious if changing -1 to other value would be enough or you propose to add other tests.
Currently there are 5 test cases:
x zp =0, y zp = 0
x zp = -1, y zp = 0
x zp = 0, y zp = -1
x zp =-1, y zp =-1
that covers all flows in QnnBatchMatmulCanonicalize
There was a problem hiding this comment.
I think we should test on larger non-zero zero points, to verify the accuracy.
|
@elvin-n Thanks. This is going to be useful for quantized transformers. cc @anijain2305 |
* Add qnn batch_matmul operator - add support of the different out type for x86 batch_matmul * Fix code style * Add out_dtype to generic batch_matmul * Restore fixe in batch_matmul for dynamic shapes * Fix documentation for qnn.batch_matmul * Remove debug code * Modify zero point for qnn batch_matmul test
* Add qnn batch_matmul operator - add support of the different out type for x86 batch_matmul * Fix code style * Add out_dtype to generic batch_matmul * Restore fixe in batch_matmul for dynamic shapes * Fix documentation for qnn.batch_matmul * Remove debug code * Modify zero point for qnn batch_matmul test
