ds003454-mriqc: "numpy.linalg.LinAlgError: SVD did not converge"

[jbwexler]

For 2 of 24 subjects:

Node: mriqc_wf.funcMRIQC.ReportsWorkflow.PlotBrainmask
Working directory: /scratch1/03201/jbwexler/work_dir/mriqc/ds003454_sub-18/mriqc_wf/funcMRIQC/ReportsWorkflow/in_file..scratch1..03201..jbwexler..openneuro_derivatives..derivatives..mriqc..ds003454-mriqc..sourcedata..raw..sub-18..func..sub-18_task-rapm_run-01_bold.nii/PlotBrainmask

Node inputs:

colors = ['r']
cut_coords = 10
display_mode = z
in_contours =
in_file =
levels = [0.5]
out_file = bmask
saturate = False
vmax =
vmin =

Traceback (most recent call last):
File "/opt/conda/lib/python3.9/site-packages/mriqc/engine/plugin.py", line 60, in run_node
result["result"] = node.run(updatehash=updatehash)
File "/opt/conda/lib/python3.9/site-packages/nipype/pipeline/engine/nodes.py", line 524, in run
result = self._run_interface(execute=True)
File "/opt/conda/lib/python3.9/site-packages/nipype/pipeline/engine/nodes.py", line 642, in _run_interface
return self._run_command(execute)
File "/opt/conda/lib/python3.9/site-packages/nipype/pipeline/engine/nodes.py", line 750, in _run_command
raise NodeExecutionError(
nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node PlotBrainmask.

Traceback (most recent call last):
File "/opt/conda/lib/python3.9/site-packages/nipype/interfaces/base/core.py", line 398, in run
runtime = self._run_interface(runtime)
File "/opt/conda/lib/python3.9/site-packages/mriqc/interfaces/viz.py", line 94, in _run_interface
plot_segmentation(
File "/opt/conda/lib/python3.9/site-packages/mriqc/viz/utils.py", line 578, in plot_segmentation
disp = plot_anat(
File "/opt/conda/lib/python3.9/site-packages/nilearn/plotting/img_plotting.py", line 473, in plot_anat
display = plot_img(anat_img, cut_coords=cut_coords,
File "/opt/conda/lib/python3.9/site-packages/nilearn/plotting/img_plotting.py", line 264, in plot_img
display = _plot_img_with_bg(
File "/opt/conda/lib/python3.9/site-packages/nilearn/plotting/img_plotting.py", line 185, in _plot_img_with_bg
display = display_factory(display_mode)(
File "/opt/conda/lib/python3.9/site-packages/nilearn/plotting/displays/_slicers.py", line 125, in init_with_figure
cut_coords = cls.find_cut_coords(img, threshold, cut_coords)
File "/opt/conda/lib/python3.9/site-packages/nilearn/plotting/displays/_slicers.py", line 1318, in find_cut_coords
cut_coords = find_cut_slices(img,
File "/opt/conda/lib/python3.9/site-packages/nilearn/plotting/find_cuts.py", line 281, in find_cut_slices
img = reorder_img(img, resample='nearest')
File "/opt/conda/lib/python3.9/site-packages/nilearn/image/resampling.py", line 766, in reorder_img
return new_img_like(img, data, affine)
File "/opt/conda/lib/python3.9/site-packages/nilearn/image/image.py", line 756, in new_img_like
return klass(data, affine, header=header)
File "/opt/conda/lib/python3.9/site-packages/nibabel/nifti1.py", line 1758, in init
super(Nifti1Pair, self).init(dataobj,
File "/opt/conda/lib/python3.9/site-packages/nibabel/analyze.py", line 918, in init
super(AnalyzeImage, self).init(
File "/opt/conda/lib/python3.9/site-packages/nibabel/spatialimages.py", line 469, in init
self.update_header()
File "/opt/conda/lib/python3.9/site-packages/nibabel/nifti1.py", line 2034, in update_header
super(Nifti1Image, self).update_header()
File "/opt/conda/lib/python3.9/site-packages/nibabel/nifti1.py", line 1797, in update_header
super(Nifti1Pair, self).update_header()
File "/opt/conda/lib/python3.9/site-packages/nibabel/spatialimages.py", line 503, in update_header
self._affine2header()
File "/opt/conda/lib/python3.9/site-packages/nibabel/nifti1.py", line 1807, in _affine2header
hdr.set_qform(self._affine, code='unknown')
File "/opt/conda/lib/python3.9/site-packages/nibabel/nifti1.py", line 1024, in set_qform
P, S, Qs = npl.svd(R)
File "<array_function internals>", line 180, in svd
File "/opt/conda/lib/python3.9/site-packages/numpy/linalg/linalg.py", line 1648, in svd
u, s, vh = gufunc(a, signature=signature, extobj=extobj)
File "/opt/conda/lib/python3.9/site-packages/numpy/linalg/linalg.py", line 97, in _raise_linalgerror_svd_nonconvergence
raise LinAlgError("SVD did not converge")
numpy.linalg.LinAlgError: SVD did not converge

[jbwexler]

also for a few subjects in ds002994

[effigies]

Simply trying to set the qform from the sform in the image did not hit this problem. Worth re-running a subject to see if it's reliable.

[jbwexler]

I reran the two failed subjects from ds003454 and got the same errors.

[jbwexler]

Also getting this for a few subjects in ds004299. Any ideas?

[effigies]

TODO: Write patched python module to bind into container.

[effigies]

Patch: nipy/nibabel#1227

To use:

datalad install -d . -s https://github.com/effigies/nibabel.git --branch enh/catch_svd_failure code/nibabel
SINGULARITYENV_PYTHONPATH="$PWD/code/nibabel" <reproman command>

@yarikoptic, just to verify, reproman will pass through these variables? Or do we need to do something like REPROENV_SINGULARITYENV_PYTHONPATH to make sure it gets through?

[yarikoptic]

to say the truth I don't remember -- I will need to try later on. If you have env handy -- just give it a shot and see.