GPU support for ndmorph, binary morphological functions #157
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
We're waiting on cupy 9.0.0 I'll move this PR out of draft status since the alpha version of cupy 9.0.0 will be made available in the next day or two. The full (stable) release of cupy 9.0.0 is scheduled for April 2021 (see release schedule here: cupy/cupy#3891). I'm not sure if we need to hold off on merging this PR until then, it may be ok to include this in dask-image earlier for those who might want to use the cupy beta releases. |
Closed
|
Do not merge: local cupy tests are failing with It's only the tests in Details:=================================== FAILURES ===================================
______________________ test_cupy_ndmorph[binary_closing] _______________________
array = dask.array<array, shape=(10, 10), dtype=float32, chunksize=(5, 5), chunktype=cupy.ndarray>
func = <function binary_closing at 0x7febc8f21430>
@pytest.mark.cupy
@pytest.mark.parametrize("func", [
ndmorph.binary_closing,
ndmorph.binary_dilation,
ndmorph.binary_erosion,
ndmorph.binary_opening,
])
def test_cupy_ndmorph(array, func):
"""Test convolve & correlate filters with cupy input arrays."""
result = func(array)
> result.compute()
tests/test_dask_image/test_ndmorph/test_cupy_ndmorph.py:31:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/dask/base.py:281: in compute
(result,) = compute(self, traverse=False, **kwargs)
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/dask/base.py:563: in compute
results = schedule(dsk, keys, **kwargs)
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/dask/threaded.py:76: in get
results = get_async(
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/dask/local.py:487: in get_async
raise_exception(exc, tb)
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/dask/local.py:317: in reraise
raise exc
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/dask/local.py:222: in execute_task
result = _execute_task(task, data)
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/dask/core.py:121: in _execute_task
return func(*(_execute_task(a, cache) for a in args))
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/dask/optimization.py:963: in __call__
return core.get(self.dsk, self.outkey, dict(zip(self.inkeys, args)))
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/dask/core.py:151: in get
result = _execute_task(task, cache)
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/dask/core.py:121: in _execute_task
return func(*(_execute_task(a, cache) for a in args))
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/dask/core.py:121: in <genexpr>
return func(*(_execute_task(a, cache) for a in args))
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/dask/core.py:121: in _execute_task
return func(*(_execute_task(a, cache) for a in args))
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/dask/core.py:121: in <genexpr>
return func(*(_execute_task(a, cache) for a in args))
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/dask/core.py:121: in _execute_task
return func(*(_execute_task(a, cache) for a in args))
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/dask/utils.py:35: in apply
return func(*args, **kwargs)
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/cupyx/scipy/ndimage/morphology.py:358: in binary_dilation
return _binary_erosion(input, structure, iterations, mask, output,
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/cupyx/scipy/ndimage/morphology.py:174: in _binary_erosion
structure = cupy.ascontiguousarray(structure)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
a = array([[False, True, False],
[ True, True, True],
[False, True, False]])
dtype = None
def ascontiguousarray(a, dtype=None):
"""Returns a C-contiguous array.
Args:
a (cupy.ndarray): Source array.
dtype: Data type specifier.
Returns:
cupy.ndarray: If no copy is required, it returns ``a``. Otherwise, it
returns a copy of ``a``.
.. seealso:: :func:`numpy.ascontiguousarray`
"""
> return core.ascontiguousarray(a, dtype)
E TypeError: Argument 'a' has incorrect type (expected cupy.core.core.ndarray, got numpy.ndarray)
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/cupy/_creation/from_data.py:97: TypeError
______________________ test_cupy_ndmorph[binary_dilation] ______________________
array = dask.array<array, shape=(10, 10), dtype=float32, chunksize=(5, 5), chunktype=cupy.ndarray>
func = <function binary_dilation at 0x7febc8f210d0>
@pytest.mark.cupy
@pytest.mark.parametrize("func", [
ndmorph.binary_closing,
ndmorph.binary_dilation,
ndmorph.binary_erosion,
ndmorph.binary_opening,
])
def test_cupy_ndmorph(array, func):
"""Test convolve & correlate filters with cupy input arrays."""
result = func(array)
> result.compute()
tests/test_dask_image/test_ndmorph/test_cupy_ndmorph.py:31:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/dask/base.py:281: in compute
(result,) = compute(self, traverse=False, **kwargs)
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/dask/base.py:563: in compute
results = schedule(dsk, keys, **kwargs)
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/dask/threaded.py:76: in get
results = get_async(
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/dask/local.py:487: in get_async
raise_exception(exc, tb)
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/dask/local.py:317: in reraise
raise exc
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/dask/local.py:222: in execute_task
result = _execute_task(task, data)
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/dask/core.py:121: in _execute_task
return func(*(_execute_task(a, cache) for a in args))
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/dask/optimization.py:963: in __call__
return core.get(self.dsk, self.outkey, dict(zip(self.inkeys, args)))
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/dask/core.py:151: in get
result = _execute_task(task, cache)
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/dask/core.py:121: in _execute_task
return func(*(_execute_task(a, cache) for a in args))
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/dask/core.py:121: in <genexpr>
return func(*(_execute_task(a, cache) for a in args))
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/dask/core.py:121: in _execute_task
return func(*(_execute_task(a, cache) for a in args))
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/dask/utils.py:35: in apply
return func(*args, **kwargs)
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/cupyx/scipy/ndimage/morphology.py:358: in binary_dilation
return _binary_erosion(input, structure, iterations, mask, output,
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/cupyx/scipy/ndimage/morphology.py:174: in _binary_erosion
structure = cupy.ascontiguousarray(structure)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
a = array([[False, True, False],
[ True, True, True],
[False, True, False]])
dtype = None
def ascontiguousarray(a, dtype=None):
"""Returns a C-contiguous array.
Args:
a (cupy.ndarray): Source array.
dtype: Data type specifier.
Returns:
cupy.ndarray: If no copy is required, it returns ``a``. Otherwise, it
returns a copy of ``a``.
.. seealso:: :func:`numpy.ascontiguousarray`
"""
> return core.ascontiguousarray(a, dtype)
E TypeError: Argument 'a' has incorrect type (expected cupy.core.core.ndarray, got numpy.ndarray)
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/cupy/_creation/from_data.py:97: TypeError
______________________ test_cupy_ndmorph[binary_erosion] _______________________
array = dask.array<array, shape=(10, 10), dtype=float32, chunksize=(5, 5), chunktype=cupy.ndarray>
func = <function binary_erosion at 0x7febc8f21040>
@pytest.mark.cupy
@pytest.mark.parametrize("func", [
ndmorph.binary_closing,
ndmorph.binary_dilation,
ndmorph.binary_erosion,
ndmorph.binary_opening,
])
def test_cupy_ndmorph(array, func):
"""Test convolve & correlate filters with cupy input arrays."""
result = func(array)
> result.compute()
tests/test_dask_image/test_ndmorph/test_cupy_ndmorph.py:31:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/dask/base.py:281: in compute
(result,) = compute(self, traverse=False, **kwargs)
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/dask/base.py:563: in compute
results = schedule(dsk, keys, **kwargs)
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/dask/threaded.py:76: in get
results = get_async(
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/dask/local.py:487: in get_async
raise_exception(exc, tb)
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/dask/local.py:317: in reraise
raise exc
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/dask/local.py:222: in execute_task
result = _execute_task(task, data)
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/dask/core.py:121: in _execute_task
return func(*(_execute_task(a, cache) for a in args))
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/dask/optimization.py:963: in __call__
return core.get(self.dsk, self.outkey, dict(zip(self.inkeys, args)))
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/dask/core.py:151: in get
result = _execute_task(task, cache)
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/dask/core.py:121: in _execute_task
return func(*(_execute_task(a, cache) for a in args))
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/dask/core.py:121: in <genexpr>
return func(*(_execute_task(a, cache) for a in args))
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/dask/core.py:121: in _execute_task
return func(*(_execute_task(a, cache) for a in args))
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/dask/utils.py:35: in apply
return func(*args, **kwargs)
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/cupyx/scipy/ndimage/morphology.py:307: in binary_erosion
return _binary_erosion(input, structure, iterations, mask, output,
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/cupyx/scipy/ndimage/morphology.py:211: in _binary_erosion
nnz = int(cupy.count_nonzero(structure))
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/cupy/_sorting/count.py:24: in count_nonzero
return _count_nonzero(a, axis=axis)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> ???
E TypeError: Argument 'a' has incorrect type (expected <class 'cupy.core.core.ndarray'>, got <class 'numpy.ndarray'>)
cupy/core/_reduction.pyx:544: TypeError
______________________ test_cupy_ndmorph[binary_opening] _______________________
array = dask.array<array, shape=(10, 10), dtype=float32, chunksize=(5, 5), chunktype=cupy.ndarray>
func = <function binary_opening at 0x7febc8f21160>
@pytest.mark.cupy
@pytest.mark.parametrize("func", [
ndmorph.binary_closing,
ndmorph.binary_dilation,
ndmorph.binary_erosion,
ndmorph.binary_opening,
])
def test_cupy_ndmorph(array, func):
"""Test convolve & correlate filters with cupy input arrays."""
result = func(array)
> result.compute()
tests/test_dask_image/test_ndmorph/test_cupy_ndmorph.py:31:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/dask/base.py:281: in compute
(result,) = compute(self, traverse=False, **kwargs)
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/dask/base.py:563: in compute
results = schedule(dsk, keys, **kwargs)
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/dask/threaded.py:76: in get
results = get_async(
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/dask/local.py:487: in get_async
raise_exception(exc, tb)
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/dask/local.py:317: in reraise
raise exc
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/dask/local.py:222: in execute_task
result = _execute_task(task, data)
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/dask/core.py:121: in _execute_task
return func(*(_execute_task(a, cache) for a in args))
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/dask/optimization.py:963: in __call__
return core.get(self.dsk, self.outkey, dict(zip(self.inkeys, args)))
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/dask/core.py:151: in get
result = _execute_task(task, cache)
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/dask/core.py:121: in _execute_task
return func(*(_execute_task(a, cache) for a in args))
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/dask/core.py:121: in <genexpr>
return func(*(_execute_task(a, cache) for a in args))
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/dask/core.py:121: in _execute_task
return func(*(_execute_task(a, cache) for a in args))
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/dask/core.py:121: in <genexpr>
return func(*(_execute_task(a, cache) for a in args))
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/dask/core.py:121: in _execute_task
return func(*(_execute_task(a, cache) for a in args))
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/dask/utils.py:35: in apply
return func(*args, **kwargs)
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/cupyx/scipy/ndimage/morphology.py:307: in binary_erosion
return _binary_erosion(input, structure, iterations, mask, output,
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/cupyx/scipy/ndimage/morphology.py:211: in _binary_erosion
nnz = int(cupy.count_nonzero(structure))
../../anaconda3/envs/dask-image-cupy-dev/lib/python3.8/site-packages/cupy/_sorting/count.py:24: in count_nonzero
return _count_nonzero(a, axis=axis)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> ???
E TypeError: Argument 'a' has incorrect type (expected <class 'cupy.core.core.ndarray'>, got <class 'numpy.ndarray'>)
cupy/core/_reduction.pyx:544: TypeError
|
|
Looks like these are failing because the default structure element is a numpy array. |
|
The cupy tests pass locally on my machine. Given that cupy 9 can currently be installed using a beta version, I'll merge this into the main branch soon. |
Closed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Pending cupy/cupy#3907
This branch contains support for GPU array computation with cupy, for the dask-image ndmorph subpackage.
Related pull requests: #150, #151
Related issue: #133