ENH: fix Python multi-processing hang on unix

[dzenanz]

Closes #2069.

Initial (old) description:

This turns a hang into:

(pyEnv) dzenan@corista:~/nn_work_dir$ python itkMultiProcessingMONAI.py 
main pid: 2578395 140116265719616
Resampling image_0.nii pid: 2578395 140116265719616...
Resampling image_0.nii pid: 2578404 140116265719616...
Resampling image_1.nii pid: 2578404 140116265719616...
In ThreadPool::atfork_prepareIn ThreadPool::atfork_resumeTraceback (most recent call last):
  File "itkMultiProcessingMONAI.py", line 44, in <module>
    p.map(f, paths)
  File "/usr/lib/python3.8/multiprocessing/pool.py", line 364, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/usr/lib/python3.8/multiprocessing/pool.py", line 771, in get
    raise self._value
multiprocessing.pool.MaybeEncodingError: Error sending result: '[<itk.itkImagePython.itkImageUC3; proxy of <Swig Object of type 'itkImageUC3 *' at 0x7f6ee94955a0> >]'. Reason: 'TypeError("cannot pickle 'SwigPyObject' object")'
(pyEnv) dzenan@corista:~/nn_work_dir$

[blowekamp]

Am I understanding correctly that with this PR, when a fork occurs, the all threads are terminated in the parent, and new threads are recreated in both new processes? I would expect the parent keeps it's threads and the child process "abandons" the parents threads and keeps new ones.

[dzenanz]

parent keeps it's threads and the child process "abandons" the parents threads

That might be possible, but with more experimentation/effort. The only thing I know of which forks with ITK is Python multiprocessing. And I am trying to address a hang in this situation, with minimal effort. Stopping and cleaning up the thread pool before the fork greatly simplifies the reasoning.

[dzenanz]

I guess that #1948 is a blocker for this. Now waiting for that to be done before continuing work on this.

[blowekamp]

@dzenanz I think if you take the test code from #2069, and just write the image instead of returning it, the pool issue should still be reproduced. This would separate this issue from #1948, to move part of it forward.

[dzenanz]

@blowekamp you are right! Changing return to imwrite avoids pickling, thus allowing me to verify this fix works. I cleaned it up as per Hans' and Matt's suggestions. Please re-review.

Addressed

[thewtex]

🎉

[thewtex]

Applied to the release branch