Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change model caching to support local pytest runs #318

Closed
bfhealy opened this issue Feb 8, 2024 · 1 comment · Fixed by #319
Closed

Change model caching to support local pytest runs #318

bfhealy opened this issue Feb 8, 2024 · 1 comment · Fixed by #319
Labels
enhancement New feature or request

Comments

@bfhealy
Copy link
Collaborator

bfhealy commented Feb 8, 2024

Running pytest nmma/tests/*.py should ideally pass all tests locally in addition to Github Actions. Caching the svdmodels in the workflow file, e.g.

- name: Restore SVD models from cache

currently leads to the analysis pytest failing locally, since local_only is set to True in nmma/tests/analysis.py.

The cached Bu2019lm model on the Potsdam enlil machine is not necessarily up to date with gitlab/zenodo (which are also currently inconsistent, see #317). There are a couple ways we could address this issue:

  1. Point the workflow to gitlab or zenodo and download the model from there at the beginning of each workflow
  2. Include Bu2019lm.pkl and a specific filter file in the nmma/tests/data directory, using that path for the analysis test's svdmodels argument
@bfhealy bfhealy added the enhancement New feature or request label Feb 8, 2024
@sahiljhawar
Copy link
Member

The cache is not stored on Potsdam machines. These are the original models. Caches are stored on Github servers (and now I cannot see any of the caches under the Actions tab). Caching was done in order to prevent timeouts (because of Zenodo). But yes, it will make sense to add the filter specifc model file in the data directory (I am not sure why didn't I do that previously).

@bfhealy bfhealy linked a pull request Feb 9, 2024 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants