Testing Consultation Summary

[ssolson]

I had a meeting with the testing group at Sandia this week and wanted to record some of the discussion.

I met with Reed and Miranda from the group 1424. MIranda works on Pyomo (Website | Github).

Suggestions for MHKiT from the initial consultation:

1. The most immediate thing MHKiT needs to tackle is to move from nose tests to pytests bc nose tests are at their end of life and will not work with py 3.10.
2. Tests local to module. Meaning that wave tests are in the wave folder. This structural change is not needed at our current stage but as the structure of MHKiT grows it is expected that the package and thereby the test organization tends to become more abstract. Having tests in the module which the tests.
3. conda install. The team found it strange that we installed miniconda but used pip install. They were okay with the tests working that way if that is how our documentation suggests to do it, but recommended we add conda install support. If we do add conda install support they further recommended we test our package with both pip install and conda install because the packages in conda tend to diffeer from pip (pip is most up to date).
4. Separate out the API calls into separate serial tests. Currently, I have hacked together a solution to avoid calling the AWS hindcast data server too many times during testing. The team suggested I break out those specific test to run in serial on their own "build" (to use GitHub actions terminology).
5. There were further recommendations around using mocks and cashing data that is calculated/ read into memory to optimize our tests.
6. Add workflow dispatch to tests

My plan is to begin working on the ones that need attention in comming weeks in parallel with our other activities.

[ssolson]

#241 will address everything here excluding the creation of a conda package. MHKiT conda package though is not related to testing though so I will leave that to a separate issue if users have interest and close this one.