Skip to content

Commit

Permalink
Set dask to single-threaded mode in tests
Browse files Browse the repository at this point in the history
Temporary work around for a segfault most likely related to installs of
netCDF4 1.7.1 from PyPI

#139
  • Loading branch information
mx-moth committed Jul 3, 2024
1 parent e58ca87 commit 4111521
Showing 1 changed file with 43 additions and 0 deletions.
43 changes: 43 additions & 0 deletions tests/conftest.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,18 @@
"""
emsarray test suite.
.. option --dask-scheduler <scheduler>
Set the scheduler for dask.
Currently the test suite segfaults when the default scheduler is used.
The scheduler is overridden to be 'synchronous', which avoids this issue.
Use this flag to choose a different scheduler for testing fixes to the segfault.
"""
import logging
import pathlib
from unittest import mock

import dask
import pytest

import emsarray
Expand All @@ -20,6 +31,38 @@ def pytest_runtest_setup(item):
item.fixturenames.append("matplotlib_backend")


def pytest_addoption(parser):
parser.addoption(
"--dask-scheduler", type=str, action="store", default="synchronous",
help=(
"Set the dask scheduler. Valid options include `synchronous' (the default), "
"`distributed', `multiprocessing', `processes', `single-threaded', "
"`sync', `synchronous', `threading', `threads'."
))

@pytest.fixture(autouse=True, scope='session')
def disable_dask_threads(request):
"""
Currently the tests will regularly segfault while subsetting ugrid datasets.
This only happens when using the latest dependencies installed from PyPI.
Using older dependencies from PyPI or using latest dependencies from
conda-forge continues to work fine.
Disabling dask multithreading stops the issue.
This is a temporary work around while the issue is investigated.
To restore the default behaviour switch to the 'threads' scheduler:
$ pytest --dask-scheduler 'threads'
See also
--------
:option:`--dask-scheduler`
https://github.com/csiro-coasts/emsarray/issues/139
"""
dask.config.set(scheduler=request.config.option.dask_scheduler)


@pytest.fixture
def matplotlib_backend(
request: pytest.FixtureRequest,
Expand Down

0 comments on commit 4111521

Please sign in to comment.