pytest annoyances and intermittent segfault

[jcermauwedu]

Most tests apparently pass. With python 3.11.9, there are these floating around.

Error in sys.excepthook:

Original exception was:
Error in sys.excepthook:

Original exception was:
Error in sys.excepthook:

Original exception was:
Error in sys.excepthook:

Original exception was:
Error in sys.excepthook:

Original exception was:
Error in sys.excepthook:

Original exception was:
Error in sys.excepthook:

Original exception was:
Error in sys.excepthook:

Original exception was:

Intermittently, I can trigger a segfault...

$ pytest --pdb
============================================================================================= test session starts =============================================================================================
platform linux -- Python 3.11.9, pytest-8.2.0, pluggy-1.5.0
rootdir: /home/portal/src/compliance-checker
configfile: pyproject.toml
plugins: requests-mock-1.12.1, time-machine-2.14.1
collected 235 items                                                                                                                                                                                           

compliance_checker/tests/test_acdd.py ................                                                                                                                                                  [  6%]
compliance_checker/tests/test_base.py ........                                                                                                                                                          [ 10%]
compliance_checker/tests/test_cf.py ...........................................................................................                                                                         [ 48%]
compliance_checker/tests/test_cf_integration.py .................                                                                                                                                       [ 56%]
compliance_checker/tests/test_cli.py .........                                                                                                                                                          [ 60%]
compliance_checker/tests/test_feature_detection.py ...............................                                                                                                                      [ 73%]
compliance_checker/tests/test_ioos_profile.py ..........................................                                                                                                                [ 91%]
compliance_checker/tests/test_ioos_sos.py ..                                                                                                                                                            [ 91%]
compliance_checker/tests/test_protocols.py ....s                                                                                                                                                        [ 94%]
compliance_checker/tests/test_suite.py ............Fatal Python error: Segmentation fault

Current thread 0x0000784bf591e740 (most recent call first):
  File "/home/portal/src/compliance-checker/compliance_checker/cf/cf_1_6.py", line 749 in _check_valid_cf_units
  File "/home/portal/src/compliance-checker/compliance_checker/cf/cf_1_6.py", line 707 in check_units
  File "/home/portal/src/compliance-checker/compliance_checker/suite.py", line 241 in _run_check
  File "/home/portal/src/compliance-checker/compliance_checker/suite.py", line 421 in run_all
  File "/home/portal/src/compliance-checker/compliance_checker/tests/test_suite.py", line 66 in test_unicode_formatting
  File "/home/portal/miniconda3/envs/gutils_py311/lib/python3.11/unittest/case.py", line 579 in _callTestMethod
  File "/home/portal/miniconda3/envs/gutils_py311/lib/python3.11/unittest/case.py", line 623 in run
  File "/home/portal/miniconda3/envs/gutils_py311/lib/python3.11/unittest/case.py", line 678 in __call__
  File "/home/portal/miniconda3/envs/gutils_py311/lib/python3.11/site-packages/_pytest/unittest.py", line 343 in runtest
  File "/home/portal/miniconda3/envs/gutils_py311/lib/python3.11/site-packages/_pytest/runner.py", line 173 in pytest_runtest_call
  File "/home/portal/miniconda3/envs/gutils_py311/lib/python3.11/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "/home/portal/miniconda3/envs/gutils_py311/lib/python3.11/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/home/portal/miniconda3/envs/gutils_py311/lib/python3.11/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "/home/portal/miniconda3/envs/gutils_py311/lib/python3.11/site-packages/_pytest/runner.py", line 241 in <lambda>
  File "/home/portal/miniconda3/envs/gutils_py311/lib/python3.11/site-packages/_pytest/runner.py", line 341 in from_call
  File "/home/portal/miniconda3/envs/gutils_py311/lib/python3.11/site-packages/_pytest/runner.py", line 240 in call_and_report
  File "/home/portal/miniconda3/envs/gutils_py311/lib/python3.11/site-packages/_pytest/runner.py", line 135 in runtestprotocol
  File "/home/portal/miniconda3/envs/gutils_py311/lib/python3.11/site-packages/_pytest/runner.py", line 116 in pytest_runtest_protocol
  File "/home/portal/miniconda3/envs/gutils_py311/lib/python3.11/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "/home/portal/miniconda3/envs/gutils_py311/lib/python3.11/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/home/portal/miniconda3/envs/gutils_py311/lib/python3.11/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "/home/portal/miniconda3/envs/gutils_py311/lib/python3.11/site-packages/_pytest/main.py", line 364 in pytest_runtestloop
  File "/home/portal/miniconda3/envs/gutils_py311/lib/python3.11/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "/home/portal/miniconda3/envs/gutils_py311/lib/python3.11/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/home/portal/miniconda3/envs/gutils_py311/lib/python3.11/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "/home/portal/miniconda3/envs/gutils_py311/lib/python3.11/site-packages/_pytest/main.py", line 339 in _main
  File "/home/portal/miniconda3/envs/gutils_py311/lib/python3.11/site-packages/_pytest/main.py", line 285 in wrap_session
  File "/home/portal/miniconda3/envs/gutils_py311/lib/python3.11/site-packages/_pytest/main.py", line 332 in pytest_cmdline_main
  File "/home/portal/miniconda3/envs/gutils_py311/lib/python3.11/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "/home/portal/miniconda3/envs/gutils_py311/lib/python3.11/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/home/portal/miniconda3/envs/gutils_py311/lib/python3.11/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "/home/portal/miniconda3/envs/gutils_py311/lib/python3.11/site-packages/_pytest/config/__init__.py", line 178 in main
  File "/home/portal/miniconda3/envs/gutils_py311/lib/python3.11/site-packages/_pytest/config/__init__.py", line 206 in console_main
  File "/home/portal/miniconda3/envs/gutils_py311/bin/pytest", line 10 in <module>

Extension modules: _brotli, simplejson._speedups, _time_machine, numpy.core._multiarray_umath, numpy.core._multiarray_tests, numpy.linalg._umath_linalg, numpy.fft._pocketfft_internal, numpy.random._common, numpy.random.bit_generator, numpy.random._bounded_integers, numpy.random._mt19937, numpy.random.mtrand, numpy.random._philox, numpy.random._pcg64, numpy.random._sfc64, numpy.random._generator, cftime._cftime, netCDF4._netCDF4, cf_units._udunits2, lxml._elementpath, lxml.etree, regex._regex, pyproj._compat, pyproj._datadir, pyproj._network, pyproj._geod, pyproj.list, pyproj._crs, pyproj.database, pyproj._transformer, pyproj._sync, shapely.lib, shapely._geos, shapely._geometry_helpers, markupsafe._speedups (total: 35)
Segmentation fault (core dumped)

[benjwadams]

I tried fixing these by suppressing the destructor in some previous commits. I've been unable to determine how exactly these are being called in such a way where I can stop these errors from appearing on the CI tests.

[ocefpaf]

I tried fixing these by suppressing the destructor in some previous commits. I've been unable to determine how exactly these are being called in such a way where I can stop these errors from appearing on the CI tests.

Most of them are due to caching on top of caching that are not release. Some, if not all, are not needed b/c the underlying slow/"called multiple times" function is already cached upstream. I'd love a speed test to be 100% sure we are not losing speed though. See #1069.

There are a few others due to how the mocked netcdf Dataset is built. I've been unable to get rid of those. Investigating more... But you can see them in action when you call self.cf.setup in the tests.

[ocefpaf]

@jcermauwedu I never experienced a segfault like that but do you mind testing against #1069 to see if that solves your problem?

[jcermauwedu]

A stress test yielded no more segfaults. If another one appears, we will let you know.

Using the latest commit, the output is squeaky clean. Thanks @benjwadams @ocefpaf

$ pytest
======================================================================== test session starts =========================================================================
platform linux -- Python 3.11.9, pytest-8.2.0, pluggy-1.5.0
rootdir: /home/portal/src/compliance-checker
configfile: pyproject.toml
plugins: cov-5.0.0, requests-mock-1.12.1, vcr-1.0.2, time-machine-2.14.1
collected 235 items                                                                                                                                                  

compliance_checker/tests/test_acdd.py ................                                                                                                         [  6%]
compliance_checker/tests/test_base.py ........                                                                                                                 [ 10%]
compliance_checker/tests/test_cf.py ...........................................................................................                                [ 48%]
compliance_checker/tests/test_cf_integration.py .................                                                                                              [ 56%]
compliance_checker/tests/test_cli.py .........                                                                                                                 [ 60%]
compliance_checker/tests/test_feature_detection.py ...............................                                                                             [ 73%]
compliance_checker/tests/test_ioos_profile.py ..........................................                                                                       [ 91%]
compliance_checker/tests/test_ioos_sos.py ..                                                                                                                   [ 91%]
compliance_checker/tests/test_protocols.py ....s                                                                                                               [ 94%]
compliance_checker/tests/test_suite.py .............                                                                                                           [ 99%]
compliance_checker/tests/test_util.py .                                                                                                                        [100%]

============================================================= 234 passed, 1 skipped in 70.58s (0:01:10) ==============================================================