Poor Performance on nq model (TF=3434ms, ONNX=6226ms)

[TomWildenhain-Microsoft]

Describe the bug
On the nq model here https://tfhub.dev/nascarr/nq/1 ONNX is significantly slower than TF. The converted model (converted with tf2onnx) takes 6226 ms per inference while TF takes 3434 ms. I'm running on CPU.

Urgency
Medium. Required for OKR target.

System information

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Windows
• ONNX Runtime installed from (source or binary): Binary (pypi)
• ONNX Runtime version: 1.5.2
• Python version: 3.8.6
• Visual Studio version (if applicable): 2019

To Reproduce

• Download https://www.dropbox.com/s/1gczlx3jn5vlx4a/perf_test_nq_portable.zip?dl=1
• Run perf_test_nq_portable.py

Expected behavior
ONNX is faster than TF

[xadupre]

Profiling with master branch:

[stale]

This issue has been automatically marked as stale due to inactivity and will be closed in 7 days if no further activity occurs. If further support is needed, please provide an update and/or more details.