Default layout inference pass

[spacycoder]

Hi, I get these warnings when running an int8 quantized model on I.MX 8M Plus using the vx delegate:

W [HandleLayoutInfer:332]Op 162: default layout inference pass.
W [HandleLayoutInfer:332]Op 56: default layout inference pass.

What does these warnings mean?

The model performs worse on NPU than CPU which I'm guessing might be related to these warning

[sergiomsilva]

Hi @spacycoder , have you figured out? I'm having the same warnings and performance issues. It seems related to matmul operations inside a custom layer I have.

[spacycoder]

No I haven't, but I also suspect it has something to do with matmul and possibly reshape ops

[sunshinemyson]

@spacycoder @sergiomsilva ,

"default layout inference pass" means that operation is not supported by layout inference framework - additional transpose operation may insert before/after your operation.

It could happen if the operation is customized op or reshape operation.

Thanks

[BralSLA]

@spacycoder @sergiomsilva ,

"default layout inference pass" means that operation is not supported by layout inference framework - additional transpose operation may insert before/after your operation.

It could happen if the operation is customized op or reshape operation.

Thanks

@sunshinemyson Currently facing the same issue. Is this something that can be solved using a different version of the vx_delegate? Would like to not rely on changing the model architecture if possible. Thanks.

[hgaiser]

I got the exact same problem, a bunch of these warnings and the performance is worse compared to just CPU. Has anyone figured out what causes this?