-
Notifications
You must be signed in to change notification settings - Fork 64
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Testing updates #494
Merged
Merged
Testing updates #494
Changes from 10 commits
Commits
Show all changes
14 commits
Select commit
Hold shift + click to select a range
43031fe
initial stab at testing-only PR
1683be3
add latest changes
b085139
fix grammar
a308887
remove tests that should be in constituents branch
a1aaf9e
remove testing files
4247019
update regexes in state_machine
104e18f
fix doctest
bbdf0d0
add state_machine updates
c2ffb05
merge up to head of feature/capgen
e75e14a
simplify workflow
d58face
remove unnecessary logic from workflow
b006850
merge up
07828e6
update metadata parser test with cleanup
1bacf44
fix merge
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,17 @@ | ||
name: Capgen Unit Tests | ||
|
||
on: | ||
climbfuji marked this conversation as resolved.
Show resolved
Hide resolved
|
||
workflow_dispatch: | ||
pull_request: | ||
branches: [feature/capgen, main] | ||
|
||
jobs: | ||
unit_tests: | ||
runs-on: ubuntu-latest | ||
steps: | ||
- uses: actions/checkout@v3 | ||
- name: update repos and install dependencies | ||
run: sudo apt-get update && sudo apt-get install -y build-essential gfortran cmake python3 git | ||
- name: Run unit tests | ||
run: cd test && ./run_fortran_tests.sh | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,28 +1,50 @@ | ||
name: Python package | ||
|
||
on: [push] | ||
on: | ||
climbfuji marked this conversation as resolved.
Show resolved
Hide resolved
|
||
workflow_dispatch: | ||
pull_request: | ||
branches: [feature/capgen, main] | ||
|
||
jobs: | ||
build: | ||
|
||
runs-on: ubuntu-latest | ||
strategy: | ||
matrix: | ||
python-version: [3.7] | ||
python-version: ['3.7', '3.8', '3.9', '3.10', '3.11'] | ||
|
||
steps: | ||
- uses: actions/checkout@v2 | ||
- uses: actions/checkout@v3 | ||
- name: Set up Python ${{ matrix.python-version }} | ||
uses: actions/setup-python@v2 | ||
uses: actions/setup-python@v4 | ||
with: | ||
python-version: ${{ matrix.python-version }} | ||
- name: Install dependencies | ||
run: | | ||
python -m pip install --upgrade pip | ||
pip install flake8 pytest | ||
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi | ||
pip install pytest | ||
- name: Test with pytest | ||
if: github.repository == 'NCAR/ccpp-framework' # Only run on main repo | ||
run: | | ||
export PYTHONPATH=$(pwd)/scripts:$(pwd)/scripts/parse_tools | ||
pytest | ||
pytest -v | ||
|
||
doctest: | ||
if: github.event_name == 'pull_request' || github.event_name == 'workflow_dispatch' || github.repository == 'NCAR/ccpp-framework' | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Do we still need the complicated logic here? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. good catch! removed. |
||
runs-on: ubuntu-latest | ||
strategy: | ||
matrix: | ||
python-version: ['3.7', '3.8', '3.9', '3.10', '3.11'] | ||
|
||
steps: | ||
- uses: actions/checkout@v3 | ||
- name: Set up Python ${{ matrix.python-version }} | ||
uses: actions/setup-python@v4 | ||
with: | ||
python-version: ${{ matrix.python-version }} | ||
- name: Install dependencies | ||
run: | | ||
python -m pip install --upgrade pip | ||
pip install pytest | ||
- name: Doctest | ||
run: | | ||
export PYTHONPATH=$(pwd)/scripts:$(pwd)/scripts/parse_tools | ||
pytest -v scripts/ --doctest-modules |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,4 +1,2 @@ | ||
[pytest] | ||
addopts = -ra -q --ignore=tests/test_capgen.py | ||
testpaths = | ||
tests | ||
addopts = -ra --ignore=scripts/metadata2html.py --ignore-glob=test/**/test_reports.py |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@gold2718 @climbfuji @mwaxmonsky @peverwhee
It would be nice if we had a CI test in the feature_capgen branch that ran the ccpp_prebuild.
We do run ccpp_prebuild as part of the ccpp-scm and ccpp-physics CI (e.g. https://github.com/ufs-community/ccpp-physics/blob/ufs/dev/.github/workflows/ci_scm_ccpp_prebuild.yml). One could easily modify this to run ccpp_prebuild using the feature_capgen fork, which would let us know if capgen development breaks the existing prebuild.
Thoughts? I can add this test to this PR if there is interest. Personally, I'd like to see this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I personally think this is a great idea! Our original thought was that if we could "unify" the testing system first then it would make unifying the actual framework easier. Of course @peverwhee and @mwaxmonsky are the ones actually responsible for all of this so I'm happy to let them make the final decision (at least for the NCAR side).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Note that we already have
ccpp_prebuild.py
tests inmain
:stub
to build aCCPP
stub that exercises the framework. SeeREADME.md
in this directory. Annoyingly, this seems to be broken right now, at least on my macOS (I'll look into fixing this later today).tests
(not thes
at the end, thetest
directory is forcapgen
). These can be run viaThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have a local workflow file that can do this similar to ccpp-physics but after discussing with @peverwhee, we are going to add that in a separate PR because it fails in the current state but successfully runs when merged with the main branch.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
created issue #500 for prebuild CI integration