Series split into multiple NIfTI volumes unexpectedly

[fedorov]

Describe the bug

While converting a CT series, conversion produces the following output, and generates 2 NIfTI files. Console output seems to suggest that converter is using InstanceNumber for ordering the slices. It does not appear possible to force the converter to reconstruct the volume using only ImageOrientationPatient/ImagePositionPatient.

The same series can be opened in 3D Slicer and viewed in OHIF viewer: https://viewer.imaging.datacommons.cancer.gov/viewer/1.2.840.113654.2.55.325606285867885853471999374864386972625?SeriesInstanceUID=1.2.840.113654.2.55.332967611868866852816434039713239600967 (including MPR) without issues.

To reproduce

Step through the cells of this very short notebook: https://colab.research.google.com/drive/1nRq9GtGiDdsX5Z7SdoyXIOR8szliNSha?usp=sharing. Data is publicly available.

Expected behavior

A single NIfTI file is produced.

Output log

Chris Rorden's dcm2niiX version v1.0.20230411  (JP2:OpenJPEG) (JP-LS:CharLS) GCC8.4.0 x86-64 (64-bit Linux)
Found 158 DICOM file(s)
Warning: PatientOrient (0018,5100) not specified (issue 642).
Warning: Instance Number (0020,0013) order is not spatial.
Warning: Interslice distance varies in this volume (incompatible with NIfTI format).
Warning: Missing images? Expected 112 images, but instance number (0020,0013) ranges from 158 to 1
Convert 112 DICOM as ./test_series/1.2.840.113654.2.55.332967611868866852816434039713239600967_1,OPA,GE,LS16,STANDARD,370,2.5,140,60,0.1,1.4_20000102000000_3 (512x512x112x1)
Warning: PatientOrient (0018,5100) not specified (issue 642).
Warning: Instance Number (0020,0013) order is not spatial.
Warning: Interslice distance varies in this volume (incompatible with NIfTI format).
Warning: Missing images? Expected 46 images, but instance number (0020,0013) ranges from 73 to 8
Convert 46 DICOM as ./test_series/1.2.840.113654.2.55.332967611868866852816434039713239600967_1,OPA,GE,LS16,STANDARD,370,2.5,140,60,0.1,1.4_20000102000000_3a (512x512x46x1)
Conversion required 6.711062 seconds (6.549128 for core code).

Version

Please report the complete version string:

Chris Rorden's dcm2niiX version v1.0.20230411 (JP2:OpenJPEG) (JP-LS:CharLS) GCC8.4.0 x86-64 (64-bit Linux)

Troubleshooting

Please try the following steps to resolve your issue:

• Is this the latest stable release? If not, does the latest stable release resolve your issue?
  • yes
• If the latest stable version fails, and you are using Windows. Does the latest commit on the development branch resolve your issue? You can get a pre-compiled version from AppVeyor (click on the Artifacts button).
  • reproducible on Mac and Linux
• If the latest stable version fails, and you are using macOS or Linux. Does the latest commit on the development branch resolve your issue?
  • no, I am getting the same error

Might be related to this discussion: #622 (comment)

[fedorov]

Also, looking at the warnings:

Warning: PatientOrient (0018,5100) not specified (issue 642).

(0018,5100) is PatientPosition (and there is no PatientOrient) https://dicom.innolitics.com/ciods/ct-image/general-series/00185100 - that warning message might need to be revisited.

[neurolabusc]

@fedorov this reflects an issue with the DICOM images. The Image Type (0008,0008) for some images is DERIVED\PRIMARY\AXIAL while for others it is ORIGINAL\PRIMARY\AXIAL. Perhaps a tool that created derived images failed to generate unique Media Object Instance UIDs (0002,0003) for the derived images, and a subsequent tool overwrote some original images with some derived data, converting two distinct images to a jumbled image that mixes some slices from one image with some slices from another image. You can make a custom fork of dcm2niix to handle these mangled series by adding a break:

case kImageTypeTag: {
    break;
    bool is1st = strlen(d.imageType) == 0;

However, this will lead to unintended consequences for situations where the derived images have modified the signal (in your case, I can not see a difference between derived and original data).

I would suggest you push this upstream to make sure that OHIF viewer does not co-mingle derived and original data.

[fedorov]

@neurolabusc thanks for the investigation and for explaining the logic. We missed that difference in ImageType!

I do not think we want to make a custom fork. The data in question is from Imaging Data Commons, and the conversion is done as part of processing to generate segmentations to be deposited back to IDC. Our goal is to implement the analysis using off-the-shelf tools and establish/test processing pipeline for DICOM data in IDC. dcm2niix is one of the tools we recommend to the users. We absolutely do not want to recommend a fork, especially since in this particular case the existing behavior makes a lot of sense!

Fortunately, a relatively small subset of images in IDC is affected by this specific issue (see ImagingDataCommons/CloudSegmentator#63), and now that we know its cause, we will be able to filter out those volumes.

I would suggest you push this upstream to make sure that OHIF viewer does not co-mingle derived and original data.

That's an interesting point - looping in @pieper @sedgi @lassoan @dclunie.

As a few final suggestions:

• it would be helpful if the console log gave some indication that ImageType inconsistency was indeed the reason for splitting the volume
• you might consider a mode where the user could override certain rules from the command line

[neurolabusc]

@fedorov the console does warn that the ImageType varies, just make sure to run it in vebose (-v 1) or logorrheic (-v 2) rather than silent (-v 0, default) mode:

dcm2niix -v 1 ~/neuro/issue786
....
Volumes not stacked: derived varies.
...

If you run dcm2niix without any arguments, it will generate help. It does list several modes. However, the modes are limited to a few common edge cases. I have never heard of a situation where one wants to intentionally combine original and derived data. If a user wants to make such a change, they could always use dcmodify (or the remove feature of gdcmanon) to make remove differences that distinguish images.

-m : merge 2D slices from same series regardless of echo, exposure, etc. (n/y or 0/1/2, default 2) [no, yes, auto]
 -s : single file mode, do not convert other images in folder (y/n, default n)

[fedorov]

Volumes not stacked: derived varies.

This is interesting, but I dare say, it is going to be very hard to figure out from that message what is actually wrong with the image (that this is about ImageType inconsistencies). Especially for those who are not intimately familiar with DICOM (which I am afraid is the case for most of users). IMHO, it would be a lot more helpful if there was a reference to the specific attribute. In the same vein, the reference/mention of PatientOrient that I mentioned earlier is not very helpful.

Just my 2c, as they say.

[neurolabusc]

@fedorov I agree, but manufacturers are not very consistent about where or even if they report derived data. For example, GE raw fieldmaps also create a fieldmap in Hz, Siemens MR can create derived diffusion images (FA, coloredFA, etc), it can be reported in image text, or a Segmentation Storage. dcm2niix can give clearer messages if the manufactuers harmonize their work, so perhaps you want to coordinate with the appropriate DICOM working group. dcm2niix is used a lot in MR brain imaging, so it is tuned to handle a lot of the data seen in that field and report anomalies common in that area. It has less exposure to other use cases, so may not be the ideal tool for beyond its domain.

[fedorov]

@neurolabusc first of all, thank you again for creating and maintaining this wonderful tool! I just wanted to share feedback as a user - thank you for considering!

I wish I had the powers to see manufacturers better harmonize their work. I do not think DICOM working groups have the intent or power to do that either, but I am tagging David Clunie @dclunie for awareness.

IMHO, your tool has a lot of value beyond the brain domain!