[tests/onnx] Add onnx and model out check

[felixdittrich92]

This PR:

• extend the test with missing allclose check for onnx export - non critical - only warn
• rs will output also why tests was skipped (only as information for us)

[felixdittrich92]

CC @llFireHawkll

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 95.96%. Comparing base (5568612) to head (c7bb257).
Report is 2 commits behind head on main.

❗ Current head c7bb257 differs from pull request most recent head d7d33c8. Consider uploading reports for the commit d7d33c8 to get more accurate results

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1569      +/-   ##
==========================================
+ Coverage   95.86%   95.96%   +0.09%     
==========================================
  Files         163      163              
  Lines        7649     7655       +6     
==========================================
+ Hits         7333     7346      +13     
+ Misses        316      309       -7     

Flag	Coverage Δ
unittests	95.96% <ø> (+0.09%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[odulcy-mindee]

I find the output misleading: https://github.com/mindee/doctr/actions/runs/8875628926/job/24365456870?pr=1569

=========================== short test summary info ============================
SKIPPED [1] tests/pytorch/test_models_detection_pt.py:183: Output of db_mobilenet_v3_large:
Difference is 9.999999747378752e-06
SKIPPED [2] tests/pytorch/test_models_recognition_pt.py:116: too less memory
=========== 235 passed, 3 skipped, 30 warnings in 302.91s (0:05:02) ============

Value seems really tiny but it's a mean and not the maximum difference observed, as computed before.
Maybe you can write as message that the outputs are not element-wise equal within a tolerance and then provide this mean difference or return the max diff you observe between two arrays.

BTW, why change the results of this test as skipped ?

[felixdittrich92]

I find the output misleading: https://github.com/mindee/doctr/actions/runs/8875628926/job/24365456870?pr=1569

=========================== short test summary info ============================
SKIPPED [1] tests/pytorch/test_models_detection_pt.py:183: Output of db_mobilenet_v3_large:
Difference is 9.999999747378752e-06
SKIPPED [2] tests/pytorch/test_models_recognition_pt.py:116: too less memory
=========== 235 passed, 3 skipped, 30 warnings in 302.91s (0:05:02) ============

Value seems really tiny but it's a mean and not the maximum difference observed, as computed before. Maybe you can write as message that the outputs are not element-wise equal within a tolerance and then provide this mean difference or return the max diff you observe between two arrays.

BTW, why change the results of this test as skipped ?

Yeah we can update the message :)
skipped is only in this special case (because we don't want that it fails here) otherwise the test will still fail or pass as before :)

Max diff is a good idea 👍

[felixdittrich92]

@odulcy-mindee missed to add 1 line xD Now you can approve and merge ^^

[llFireHawkll]

Great thanks @felixdittrich92 for this update. Really appreciate your help.