Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Testing updates #494

Merged
merged 14 commits into from
Oct 12, 2023
Merged
Show file tree
Hide file tree
Changes from 10 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions .github/workflows/capgen_unit_tests.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
name: Capgen Unit Tests
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gold2718 @climbfuji @mwaxmonsky @peverwhee
It would be nice if we had a CI test in the feature_capgen branch that ran the ccpp_prebuild.

We do run ccpp_prebuild as part of the ccpp-scm and ccpp-physics CI (e.g. https://github.com/ufs-community/ccpp-physics/blob/ufs/dev/.github/workflows/ci_scm_ccpp_prebuild.yml). One could easily modify this to run ccpp_prebuild using the feature_capgen fork, which would let us know if capgen development breaks the existing prebuild.
Thoughts? I can add this test to this PR if there is interest. Personally, I'd like to see this.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I personally think this is a great idea! Our original thought was that if we could "unify" the testing system first then it would make unifying the actual framework easier. Of course @peverwhee and @mwaxmonsky are the ones actually responsible for all of this so I'm happy to let them make the final decision (at least for the NCAR side).

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that we already have ccpp_prebuild.py tests in main:

  1. In stub to build a CCPP stub that exercises the framework. See README.md in this directory. Annoyingly, this seems to be broken right now, at least on my macOS (I'll look into fixing this later today).
  2. In subdirectory tests (not the s at the end, the test directory is for capgen). These can be run via
cd tests
PYTHONPATH="$PWD/../scripts:$PWD/../scripts/parse_tools:$PYTHONPATH" python3 test_mkstatic.py
PYTHONPATH="$PWD/../scripts:$PWD/../scripts/parse_tools:$PYTHONPATH" python3 test_metadata_parser.py

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a local workflow file that can do this similar to ccpp-physics but after discussing with @peverwhee, we are going to add that in a separate PR because it fails in the current state but successfully runs when merged with the main branch.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

created issue #500 for prebuild CI integration


on:
climbfuji marked this conversation as resolved.
Show resolved Hide resolved
workflow_dispatch:
pull_request:
branches: [feature/capgen, main]

jobs:
unit_tests:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: update repos and install dependencies
run: sudo apt-get update && sudo apt-get install -y build-essential gfortran cmake python3 git
- name: Run unit tests
run: cd test && ./run_fortran_tests.sh

40 changes: 31 additions & 9 deletions .github/workflows/python.yaml
Original file line number Diff line number Diff line change
@@ -1,28 +1,50 @@
name: Python package

on: [push]
on:
climbfuji marked this conversation as resolved.
Show resolved Hide resolved
workflow_dispatch:
pull_request:
branches: [feature/capgen, main]

jobs:
build:

runs-on: ubuntu-latest
strategy:
matrix:
python-version: [3.7]
python-version: ['3.7', '3.8', '3.9', '3.10', '3.11']

steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install flake8 pytest
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
pip install pytest
- name: Test with pytest
if: github.repository == 'NCAR/ccpp-framework' # Only run on main repo
run: |
export PYTHONPATH=$(pwd)/scripts:$(pwd)/scripts/parse_tools
pytest
pytest -v

doctest:
if: github.event_name == 'pull_request' || github.event_name == 'workflow_dispatch' || github.repository == 'NCAR/ccpp-framework'
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we still need the complicated logic here?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch! removed.

runs-on: ubuntu-latest
strategy:
matrix:
python-version: ['3.7', '3.8', '3.9', '3.10', '3.11']

steps:
- uses: actions/checkout@v3
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install pytest
- name: Doctest
run: |
export PYTHONPATH=$(pwd)/scripts:$(pwd)/scripts/parse_tools
pytest -v scripts/ --doctest-modules
4 changes: 1 addition & 3 deletions pytest.ini
Original file line number Diff line number Diff line change
@@ -1,4 +1,2 @@
[pytest]
addopts = -ra -q --ignore=tests/test_capgen.py
testpaths =
tests
addopts = -ra --ignore=scripts/metadata2html.py --ignore-glob=test/**/test_reports.py
27 changes: 9 additions & 18 deletions scripts/code_block.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
class CodeBlock(object):
"""Class to store a block of code and a method to write it to a file
>>> CodeBlock([]) #doctest: +ELLIPSIS
<__main__.CodeBlock object at 0x...>
<code_block.CodeBlock object at 0x...>
>>> CodeBlock(['hi mom']) #doctest: +IGNORE_EXCEPTION_DETAIL
Traceback (most recent call last):
ParseInternalError: Each element of <code_list> must contain exactly two items, a code string and a relative indent
Expand All @@ -24,14 +24,21 @@ class CodeBlock(object):
Traceback (most recent call last):
ParseInternalError: Each element of <code_list> must contain exactly two items, a code string and a relative indent
>>> CodeBlock([('hi mom', 1)]) #doctest: +ELLIPSIS
<__main__.CodeBlock object at 0x...>
<code_block.CodeBlock object at 0x...>
>>> from fortran_tools import FortranWriter
>>> outfile_name = "__code_block_temp.F90"
>>> outfile = FortranWriter(outfile_name, 'w', 'test file', 'test_mod')
>>> CodeBlock([('hi mom', 1)]).write(outfile, 1, {})

>>> CodeBlock([('hi {greet} mom', 1)]).write(outfile, 1, {}) #doctest: +IGNORE_EXCEPTION_DETAIL
Traceback (most recent call last):
ParseInternalError: 'greet' missing from <var_dict>
>>> CodeBlock([('hi {{greet}} mom', 1)]).write(outfile, 1, {})
>>> CodeBlock([('{greet} there mom', 1)]).write(outfile, 1, {'greet':'hi'})
>>> outfile.__exit__()
False
>>> import os
>>> os.remove(outfile_name)
"""

__var_re = re.compile(r"[{][ ]*([A-Za-z][A-Za-z0-9_]*)[ ]*[}]")
Expand Down Expand Up @@ -110,19 +117,3 @@ def write(self, outfile, indent_level, var_dict):
# end for

###############################################################################
if __name__ == "__main__":
# pylint: disable=ungrouped-imports
import doctest
import os
import sys
from fortran_tools import FortranWriter
# pylint: enable=ungrouped-imports
outfile_name = "__code_block_temp.F90"
with FortranWriter(outfile_name, 'w', 'test file', 'test_mod') as outfile:
fail, _ = doctest.testmod()
# end with
if os.path.exists(outfile_name):
os.remove(outfile_name)
# end if
sys.exit(fail)
# end if
16 changes: 4 additions & 12 deletions scripts/fortran_tools/parse_fortran.py
Original file line number Diff line number Diff line change
Expand Up @@ -665,6 +665,10 @@ def parse_fortran_var_decl(line, source, run_env):
'(8)'
>>> _VAR_ID_RE.match("foo(::,a:b,a:,:b)").group(2)
'(::,a:b,a:,:b)'
>>> from framework_env import CCPPFrameworkEnv
>>> _DUMMY_RUN_ENV = CCPPFrameworkEnv(None, ndict={'host_files':'', \
'scheme_files':'', \
'suites':''})
>>> parse_fortran_var_decl("integer :: foo", ParseSource('foo.F90', 'module', ParseContext()), _DUMMY_RUN_ENV)[0].get_prop_value('local_name')
'foo'
>>> parse_fortran_var_decl("integer :: foo = 0", ParseSource('foo.F90', 'module', ParseContext()), _DUMMY_RUN_ENV)[0].get_prop_value('local_name')
Expand Down Expand Up @@ -826,15 +830,3 @@ def parse_fortran_var_decl(line, source, run_env):
########################################################################

########################################################################

if __name__ == "__main__":
# pylint: disable=ungrouped-imports
import doctest
# pylint: enable=ungrouped-imports
from framework_env import CCPPFrameworkEnv
_DUMMY_RUN_ENV = CCPPFrameworkEnv(None, ndict={'host_files':'',
'scheme_files':'',
'suites':''})
fail, _ = doctest.testmod()
sys.exit(fail)
# end if
18 changes: 5 additions & 13 deletions scripts/metadata_table.py
Original file line number Diff line number Diff line change
Expand Up @@ -504,14 +504,18 @@ def table_start(cls, line):

class MetadataSection(ParseSource):
"""Class to hold all information from a metadata header
>>> from framework_env import CCPPFrameworkEnv
>>> _DUMMY_RUN_ENV = CCPPFrameworkEnv(None, {'host_files':'', \
'scheme_files':'', \
'suites':''})
>>> MetadataSection("footable", "scheme", _DUMMY_RUN_ENV, \
parse_object=ParseObject("foobar.txt", \
["name = footable", "type = scheme", "module = foo", \
"[ im ]", "standard_name = horizontal_loop_extent", \
"long_name = horizontal loop extent, start at 1", \
"units = index | type = integer", \
"dimensions = () | intent = in"])) #doctest: +ELLIPSIS
<__main__.MetadataSection foo / footable at 0x...>
<metadata_table.MetadataSection foo / footable at 0x...>
>>> MetadataSection("footable", "scheme", _DUMMY_RUN_ENV, \
parse_object=ParseObject("foobar.txt", \
["name = footable", "type = scheme", "module = foobar", \
Expand Down Expand Up @@ -1310,15 +1314,3 @@ def is_scalar_reference(test_val):
return check_fortran_ref(test_val, None, False) is not None

########################################################################

if __name__ == "__main__":
# pylint: enable=ungrouped-imports
import doctest
import sys
# pylint: disable=ungrouped-imports
from framework_env import CCPPFrameworkEnv
_DUMMY_RUN_ENV = CCPPFrameworkEnv(None, {'host_files':'',
'scheme_files':'',
'suites':''})
fail, _ = doctest.testmod()
sys.exit(fail)
12 changes: 2 additions & 10 deletions scripts/metavar.py
Original file line number Diff line number Diff line change
Expand Up @@ -1379,11 +1379,11 @@ class VarDictionary(OrderedDict):
>>> VarDictionary('bar', _MVAR_DUMMY_RUN_ENV, variables={})
VarDictionary(bar)
>>> VarDictionary('baz', _MVAR_DUMMY_RUN_ENV, variables=Var({'local_name' : 'foo', 'standard_name' : 'hi_mom', 'units' : 'm s-1', 'dimensions' : '()', 'type' : 'real', 'intent' : 'in'}, ParseSource('vname', 'scheme', ParseContext()), _MVAR_DUMMY_RUN_ENV)) #doctest: +ELLIPSIS
VarDictionary(baz, [('hi_mom', <__main__.Var hi_mom: foo at 0x...>)])
VarDictionary(baz, [('hi_mom', <metavar.Var hi_mom: foo at 0x...>)])
>>> print("{}".format(VarDictionary('baz', _MVAR_DUMMY_RUN_ENV, variables=Var({'local_name' : 'foo', 'standard_name' : 'hi_mom', 'units' : 'm s-1', 'dimensions' : '()', 'type' : 'real', 'intent' : 'in'}, ParseSource('vname', 'scheme', ParseContext()), _MVAR_DUMMY_RUN_ENV))))
VarDictionary(baz, ['hi_mom'])
>>> VarDictionary('qux', _MVAR_DUMMY_RUN_ENV, variables=[Var({'local_name' : 'foo', 'standard_name' : 'hi_mom', 'units' : 'm s-1', 'dimensions' : '()', 'type' : 'real', 'intent' : 'in'}, ParseSource('vname', 'scheme', ParseContext()), _MVAR_DUMMY_RUN_ENV)]) #doctest: +ELLIPSIS
VarDictionary(qux, [('hi_mom', <__main__.Var hi_mom: foo at 0x...>)])
VarDictionary(qux, [('hi_mom', <metavar.Var hi_mom: foo at 0x...>)])
>>> VarDictionary('boo', _MVAR_DUMMY_RUN_ENV).add_variable(Var({'local_name' : 'foo', 'standard_name' : 'hi_mom', 'units' : 'm s-1', 'dimensions' : '()', 'type' : 'real', 'intent' : 'in'}, ParseSource('vname', 'scheme', ParseContext()), _MVAR_DUMMY_RUN_ENV), _MVAR_DUMMY_RUN_ENV)

>>> VarDictionary('who', _MVAR_DUMMY_RUN_ENV, variables=[Var({'local_name' : 'foo', 'standard_name' : 'hi_mom', 'units' : 'm s-1', 'dimensions' : '()', 'type' : 'real', 'intent' : 'in'}, ParseSource('vname', 'scheme', ParseContext()), _MVAR_DUMMY_RUN_ENV)]).prop_list('local_name')
Expand Down Expand Up @@ -1982,11 +1982,3 @@ def new_internal_variable_name(self, prefix=None, max_len=63):
_MVAR_DUMMY_RUN_ENV)])

###############################################################################
if __name__ == "__main__":
# pylint: disable=ungrouped-imports
import doctest
import sys
# pylint: enable=ungrouped-imports
fail, _ = doctest.testmod()
sys.exit(fail)
# end if
11 changes: 1 addition & 10 deletions scripts/parse_tools/parse_object.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ class ParseObject(ParseContext):
"""ParseObject is a simple class that keeps track of an object's
place in a file and safely produces lines from an array of lines
>>> ParseObject('foobar.F90', []) #doctest: +ELLIPSIS
<__main__.ParseObject object at 0x...>
<parse_tools.parse_object.ParseObject object at 0x...>
>>> ParseObject('foobar.F90', []).filename
'foobar.F90'
>>> ParseObject('foobar.F90', ["##hi mom",], line_start=1).curr_line()
Expand Down Expand Up @@ -170,12 +170,3 @@ def __del__(self):
# end try

########################################################################

if __name__ == "__main__":
# pylint: disable=ungrouped-imports
import doctest
import sys
# pylint: enable=ungrouped-imports
fail, _ = doctest.testmod()
sys.exit(fail)
# end if
14 changes: 3 additions & 11 deletions scripts/parse_tools/parse_source.py
Original file line number Diff line number Diff line change
Expand Up @@ -201,10 +201,10 @@ def __getitem__(self, index):
class ParseContext(object):
"""A class for keeping track of a parsing position
>>> ParseContext(32, "source.F90") #doctest: +ELLIPSIS
<__main__.ParseContext object at 0x...>
<parse_tools.parse_source.ParseContext object at 0x...>
>>> ParseContext("source.F90", 32)
Traceback (most recent call last):
CCPPError: ParseContext linenum must be an int
parse_tools.parse_source.CCPPError: ParseContext linenum must be an int
>>> ParseContext(32, 90) #doctest: +IGNORE_EXCEPTION_DETAIL
Traceback (most recent call last):
CCPPError: ParseContext filenum must be a string
Expand Down Expand Up @@ -382,7 +382,7 @@ class ParseSource(object):
"""
A simple object for providing source information
>>> ParseSource("myname", "mytype", ParseContext(13, "foo.F90")) #doctest: +ELLIPSIS
<__main__.ParseSource object at 0x...>
<parse_tools.parse_source.ParseSource object at 0x...>
>>> ParseSource("myname", "mytype", ParseContext(13, "foo.F90")).type
'mytype'
>>> ParseSource("myname", "mytype", ParseContext(13, "foo.F90")).name
Expand Down Expand Up @@ -413,11 +413,3 @@ def context(self):
return self._context

########################################################################

if __name__ == "__main__":
# pylint: disable=ungrouped-imports
import doctest
# pylint: enable=ungrouped-imports
fail, _ = doctest.testmod()
sys.exit(fail)
# end if
18 changes: 2 additions & 16 deletions scripts/parse_tools/xml_tools.py
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,8 @@ def call_command(commands, logger, silent=False):
###############################################################################
"""
Try a command line and return the output on success (None on failure)
>>> _LOGGER = init_log('xml_tools')
>>> set_log_to_null(_LOGGER)
>>> call_command(['ls', 'really__improbable_fffilename.foo'], _LOGGER) #doctest: +IGNORE_EXCEPTION_DETAIL
Traceback (most recent call last):
CCPPError: Execution of 'ls really__improbable_fffilename.foo' failed:
Expand Down Expand Up @@ -350,19 +352,3 @@ def write(self, file, encoding="us-ascii", xml_declaration=None,
# end with

##############################################################################

if __name__ == "__main__":
_LOGGER = init_log('xml_tools')
set_log_to_null(_LOGGER)
try:
# First, run doctest
# pylint: disable=ungrouped-imports
import doctest
# pylint: enable=ungrouped-imports
fail, _ = doctest.testmod()
sys.exit(fail)
except CCPPError as cerr:
print("{}".format(cerr))
sys.exit(fail)
# end try
# end if
Loading