Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add eval() to remove Torch warnings during testing #472

Merged
merged 3 commits into from
Feb 2, 2024

Conversation

mellis13
Copy link
Contributor

When running the SmartSim test suite with the Torch backend installed, warnings are surfaced as shown below:

tests/backends/test_dbmodel.py::test_pt_db_model
  /Users/torch/jit/_trace.py:1084: TracerWarning: Output nr 1. of the traced function does not match the corresponding output of the Python function. Detailed error:
  Tensor-likes are not close!
  
  Mismatched elements: 10 / 10 (100.0%)
  Greatest absolute difference: 0.0451509952545166 at index (0, 9) (up to 1e-05 allowed)
  Greatest relative difference: 0.019849935548005467 at index (0, 9) (up to 1e-05 allowed)
    _check_trace(

This appears to stem from not having the eval() function invoked before jit tracing and saving the model. It appears after adding this call that warnings no longer appear.

@mellis13 mellis13 added the area: test Issues related to the test suite label Jan 31, 2024
Copy link

codecov bot commented Jan 31, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (b160c05) 90.77% compared to head (e6fe000) 90.72%.
Report is 1 commits behind head on develop.

Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff             @@
##           develop     #472      +/-   ##
===========================================
- Coverage    90.77%   90.72%   -0.06%     
===========================================
  Files           60       60              
  Lines         3837     3837              
===========================================
- Hits          3483     3481       -2     
- Misses         354      356       +2     

see 1 file with indirect coverage changes

Copy link
Member

@ashao ashao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch @mellis13, thanks for chasing it down.

Copy link
Member

@MattToast MattToast left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One VERY small comment, but nothing worth holding approval up over. LGTM!! Thanks for squashing those warnings!!

@@ -42,7 +45,7 @@ Detailed Notes
all of SmartSim's machine learning backends with Python 3.11.
(SmartSim-PR451_) (SmartSim-PR461_)


.. _SmartSim-PR472: https://github.com/CrayLabs/SmartSim/pull/472
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Incredibly minor detail, but its going to bug me if I don't mention it:

This link is the only one in this list not in sorted order (I know, incredibly pedantic, but my stupid brain needs things to match, hahaha). Can we move it to the bottom?

@mellis13 mellis13 merged commit 106d70f into CrayLabs:develop Feb 2, 2024
34 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area: test Issues related to the test suite
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants