Support for exporting BIDS compatible JSON files

[chrisgorgo]

Brain Imaging Data Structure (BIDS) is a new specification describing how a neuroimaging dataset should be organized and described. Part of the standard are JSON sidecar files with acquisition parameters essential for performing data analysis that are not present (or reliably reported) in the NIFTI header (see here for details). Such fields include but are not limited to:

• EffectiveEchoSpacing
• RepetitionTime
• PhaseEncodingDirection
• SliceTiming
• SliceEncodingDirection
• EchoTime

Some of those fields are part of DICOM Ontology and are directly accessible from standard DICOM headers (such as RepetitionTime and EchoTime) and some are not part of standard DICOM nomenclature and require extraction using vendor and sequence specific heuristics (for example PhaseEncodingDirection or EffectiveEchoSpacing). We aded them to the BIDS standard because they are necessary for data processing.

This is how an example BIDS compatible JSON file looks like (more examples here):

{
    "EchoTime": 0.017,
    "EffectiveEchoSpacing": 0.0003333262223739227,
    "PhaseEncodingDirection": "y-",
    "RepetitionTime": 3.0,
    "SliceEncodingDirection": "z",
    "SliceTiming": [
        1.508,
        0.0,
        1.55,
        0.043,
        1.592,
        0.087,
        1.635,
        0.13,
        1.677,
        0.173,
        1.722,
        0.215,
        1.765,
        0.26,
        1.808,
        0.302,
        1.85,
        0.345,
        1.893,
        0.388,
        1.938,
        0.43,
        1.98,
        0.475,
        2.022,
        0.518,
        2.065,
        0.56,
        2.11,
        0.603,
        2.152,
        0.645,
        2.195,
        0.69,
        2.238,
        0.733,
        2.28,
        0.775,
        2.325,
        0.818,
        2.367,
        0.86,
        2.41,
        0.905,
        2.453,
        0.948,
        2.495,
        0.99,
        2.54,
        1.032,
        2.583,
        1.075,
        2.625,
        1.12,
        2.668,
        1.163,
        2.71,
        1.205,
        2.755,
        1.248,
        2.798,
        1.293,
        2.84,
        1.335,
        2.883,
        1.378,
        2.925,
        1.42,
        2.97,
        1.462
    ]
}

dcm2nii has for year been the benchmark of efficient and precise DICOM to NIFTI converters. It would be great if the future release supported saving BIDS style JSON files along the converted NIFTI files. In addition you can consider embedding this JSON inside the header (similar to what dcmstack does). From looking at your code base I see that you already calculate many of the required parameters. I'm more than happy to provide any help in implementing this feature.

[neurolabusc]

I have added experimental BIDS support - I will let you test this and expand it if you feel it is worthwhile. At the moment BIDS creation is disabled by default, you enable creation of BIDS output with the "-b y", for example to convert all the DICOM files in the folder "dcm" and include BIDS support you would call
./dcm2niix -b y ~/dcm
You will want to tune the function "nii_SaveBIDS" in the unit "nii_dicom_batch.cpp" to suit your needs. As you see in my code, I created the tag "PhaseEncodingDirectionRowColumn" instead of your "PhaseEncodingDirection". Feel free to edit this if you want, but I believe you should use either the DICOM convention of referring to image space as "Rows"/"Columns" or the NIfTI convention of referring to these dimensions as "i"/"j". The terms "X"/"Y"/"Z" are reserved in NIfTI to refer to world space, not image space, e.g. "Z" is header-foot regardless of whether this was the row/column/slice of acquisition. I think compatibility with a standard is more important than compatibility between standards, so if the BIDS group insists on X/Y/Z you can edit the file to match that, but if you think of the manipulations we apply during spatial processing I do think the NIfTI/DICOM conventions are good ones.

Further, while I think the goals of BIDS are great filling a clear need, and the current scope is focused, I do worry that formats get extended to applications where they are not appropriate. In general I think text formats are a poor choice for science, due to precision, performance, file size and maintainability concerns (e.g. decimal separator, EOLN handling, case sensitivity, unicode handling, etc). Some of these basic concerns are outlined here
http://www.dursi.ca/scientific-data-shouldnt-be-written-in-text/
I worry that neuroscience is a young field, and we tend to be lured by formats that are appropriate for text-based web data rather than looking at the robust solutions that have been developed in the other sciences (e.g. HDF).

[halfSpinDoctor]

​Not to interfere too much with the conversation, but is this ​really a new
format or just a way of storing the DICOM headers in a json format, with a
slightly different dictionary (i.e. labels for fields)?

Given that there is already a program that does this well,
https://github.com/moloney/dcmstack, does it make more sense to go with the
unix philosophy and have users call dcm2niix for the actual image
conversion step, then call dcmstack separately to generate the metadata?

Not to say that it isn't good to give people options, but given that
exactly what you want already seems to be implemented, does it make sense
to have to re-write it all from Python, and then have to maintain two
codebases?

BW,
-Sam

---

Samuel A. Hurley, PhD
MRI Physics Support Scientist
FMRIB Centre, University of Oxford
Telephone: +44 (0) 1865 222733
http://users.fmrib.ox.ac.uk/~shurley/

On 17 November 2015 at 08:59, Chris Rorden notifications@github.com wrote:

I have added experimental BIDS support - I will let you test this and
expand it if you feel it is worthwhile. At the moment BIDS creation is
disabled by default, you enable creation of BIDS output with the "-b y",
for example to convert all the DICOM files in the folder "dcm" and include
BIDS support you would call
./dcm2niix -b y ~/dcm
You will want to tune the function "nii_SaveBIDS" in the unit
"nii_dicom_batch.cpp" to suit your needs. As you see in my code, I created
the tag "PhaseEncodingDirectionRowColumn" instead of your
"PhaseEncodingDirection". Feel free to edit this if you want, but I believe
you should use either the DICOM convention of referring to image space as
"Rows"/"Columns" or the NIfTI convention of referring to these dimensions
as "i"/"j". The terms "X"/"Y"/"Z" are reserved in NIfTI to refer to world
space, not image space, e.g. "Z" is header-foot regardless of whether this
was the row/column/slice of acquisition. I think compatibility with a
standard is more important than compatibility between standards, so if the
BIDS group insists on X/Y/Z you can edit the file to match that, but if you
think of the manipulations we apply during spatial processing I do think
the NIfTI/DICOM conventions are good ones.

Further, while I think the goals of BIDS are great filling a clear need,
and the current scope is focused, I do worry that formats get extended to
applications where they are not appropriate. In general I think text
formats are a poor choice for science, due to precision, performance, file
size and maintainability concerns (e.g. decimal separator, EOLN handling,
case sensitivity, unicode handling, etc). Some of these basic concerns are
outlined here
http://www.dursi.ca/scientific-data-shouldnt-be-written-in-text/
I worry that neuroscience is a young field, and we tend to be lured by
formats that are appropriate for text-based web data rather than looking at
the robust solutions that have been developed in the other sciences (e.g.
HDF).

—
Reply to this email directly or view it on GitHub
https://github.com/neurolabusc/dcm2niix/issues/4#issuecomment-157394294.

[chrisgorgo]

On Tue, Nov 17, 2015 at 6:59 AM, Chris Rorden notifications@github.com
wrote:

I have added experimental BIDS support - I will let you test this and
expand it if you feel it is worthwhile. At the moment BIDS creation is
disabled by default, you enable creation of BIDS output with the "-b y",
for example to convert all the DICOM files in the folder "dcm" and include
BIDS support you would call
./dcm2niix -b y ~/dcm

This is great! Thank you!

You will want to tune the function "nii_SaveBIDS" in the unit
"nii_dicom_batch.cpp" to suit your needs. As you see in my code, I created
the tag "PhaseEncodingDirectionRowColumn" instead of your
"PhaseEncodingDirection". Feel free to edit this if you want, but I believe
you should use either the DICOM convention of referring to image space as
"Rows"/"Columns" or the NIfTI convention of referring to these dimensions
as "i"/"j". The terms "X"/"Y"/"Z" are reserved in NIfTI to refer to world
space, not image space, e.g. "Z" is header-foot regardless of whether this
was the row/column/slice of acquisition. I think compatibility with a
standard is more important than compatibility between standards, so if the
BIDS group insists on X/Y/Z you can edit the file to match that, but if you
think of the manipulations we apply during spatial processing I do think
the NIfTI/DICOM conventions are good ones.

Well the reason why we went with "x", "x-", "y", "y-", "z", "z-" is that
simply specifying "Rows" or "Columns" is not enough to perform field
unwarping and fieldmap estimation using TOPUP (actually his nomenclature
has been inspired by the --unwarpdir argument used by FSL FUGUE). You need
the axis number (is "Rows" the first axis or the second? What's the label
for the third axis?). We could have used different labels for example "0",
"1", "2" and an extra field for positive/negative (like you did), but this
will be functionally the same, with the downside of requiring a backward
compatible change (we are at a Release Candidate stage). The meaning of
values of "PhaseEncodingDirection" field is well defined in the BIDS
specification so I don't expect any problems with interpretation. Would you
be willing to merge a patch if I submit one fixing this (based on the code
included in dicm2nii)?

Additionally if you don't want to provide this information I would
recommend to reuse DICOM terms instead of coining new ones. Rename
"PhaseEncodingDirectionRowColumn" to "InPlaneEncodingDirection" and use
"ROW"/"COL" instead of "Rows"/"Columns".

Further, while I think the goals of BIDS are great filling a clear need,
and the current scope is focused, I do worry that formats get extended to
applications where they are not appropriate. In general I think text
formats are a poor choice for science, due to precision, performance, file
size and maintainability concerns (e.g. decimal separator, EOLN handling,
case sensitivity, unicode handling, etc). Some of these basic concerns are
outlined here
http://www.dursi.ca/scientific-data-shouldnt-be-written-in-text/
I worry that neuroscience is a young field, and we tend to be lured by
formats that are appropriate for text-based web data rather than looking at
the robust solutions that have been developed in the other sciences (e.g.
HDF).

—
Reply to this email directly or view it on GitHub
https://github.com/neurolabusc/dcm2niix/issues/4#issuecomment-157394294.

@halfSpinDoctor BIDS is reusing DICOM terms whenever it is possible and
introducing new ones where there is not consistency between scanner
manufacturers (PhaseEncodingDirection, SliceTiming, EffectiveEchoSpacing
etc.), so it's role is part description part normalization. We did reach
out to dcmstack folks if they would like to include those extra non-DICOM
fields moloney/dcmstack#39, but they want to
stick with just reporting the content of DICOM headers (without normalizing
scanner specific fields). Having said that I am planning to release a tool
that will output BIDS compatible .json file based on dcmstack output.

[halfSpinDoctor]

Hi Chris,

Thanks for the clarification. That makes more sense.

This is an interesting project, and I am planning on carefully reading your
proposal on Google Doc. I would be interested ot see how other quantitative
methods such as DMRI, QMTI, etc, might also fit in. I'll comment over on
Google with any further thoughts as to not clog up this support ticket.

S

---

Samuel A. Hurley, PhD
MRI Physics Support Scientist
FMRIB Centre, University of Oxford
Telephone: +44 (0) 1865 222733
http://users.fmrib.ox.ac.uk/~shurley/

On 17 November 2015 at 10:11, Chris Filo Gorgolewski <
notifications@github.com> wrote:

On Tue, Nov 17, 2015 at 6:59 AM, Chris Rorden notifications@github.com
wrote:

I have added experimental BIDS support - I will let you test this and
expand it if you feel it is worthwhile. At the moment BIDS creation is
disabled by default, you enable creation of BIDS output with the "-b y",
for example to convert all the DICOM files in the folder "dcm" and
include
BIDS support you would call
./dcm2niix -b y ~/dcm

This is great! Thank you!

You will want to tune the function "nii_SaveBIDS" in the unit
"nii_dicom_batch.cpp" to suit your needs. As you see in my code, I
created
the tag "PhaseEncodingDirectionRowColumn" instead of your
"PhaseEncodingDirection". Feel free to edit this if you want, but I
believe
you should use either the DICOM convention of referring to image space as
"Rows"/"Columns" or the NIfTI convention of referring to these dimensions
as "i"/"j". The terms "X"/"Y"/"Z" are reserved in NIfTI to refer to world
space, not image space, e.g. "Z" is header-foot regardless of whether
this
was the row/column/slice of acquisition. I think compatibility with a
standard is more important than compatibility between standards, so if
the
BIDS group insists on X/Y/Z you can edit the file to match that, but if
you
think of the manipulations we apply during spatial processing I do think
the NIfTI/DICOM conventions are good ones.

Well the reason why we went with "x", "x-", "y", "y-", "z", "z-" is that
simply specifying "Rows" or "Columns" is not enough to perform field
unwarping and fieldmap estimation using TOPUP (actually his nomenclature
has been inspired by the --unwarpdir argument used by FSL FUGUE). You need
the axis number (is "Rows" the first axis or the second? What's the label
for the third axis?). We could have used different labels for example "0",
"1", "2" and an extra field for positive/negative (like you did), but this
will be functionally the same, with the downside of requiring a backward
compatible change (we are at a Release Candidate stage). The meaning of
values of "PhaseEncodingDirection" field is well defined in the BIDS
specification so I don't expect any problems with interpretation. Would you
be willing to merge a patch if I submit one fixing this (based on the code
included in dicm2nii)?

Additionally if you don't want to provide this information I would
recommend to reuse DICOM terms instead of coining new ones. Rename
"PhaseEncodingDirectionRowColumn" to "InPlaneEncodingDirection" and use
"ROW"/"COL" instead of "Rows"/"Columns".

Further, while I think the goals of BIDS are great filling a clear need,
and the current scope is focused, I do worry that formats get extended to
applications where they are not appropriate. In general I think text
formats are a poor choice for science, due to precision, performance,
file
size and maintainability concerns (e.g. decimal separator, EOLN handling,
case sensitivity, unicode handling, etc). Some of these basic concerns
are
outlined here
http://www.dursi.ca/scientific-data-shouldnt-be-written-in-text/
I worry that neuroscience is a young field, and we tend to be lured by
formats that are appropriate for text-based web data rather than looking
at
the robust solutions that have been developed in the other sciences (e.g.
HDF).

—
Reply to this email directly or view it on GitHub
<https://github.com/neurolabusc/dcm2niix/issues/4#issuecomment-157394294
.

@halfSpinDoctor BIDS is reusing DICOM terms whenever it is possible and
introducing new ones where there is not consistency between scanner
manufacturers (PhaseEncodingDirection, SliceTiming, EffectiveEchoSpacing
etc.), so it's role is part description part normalization. We did reach
out to dcmstack folks if they would like to include those extra non-DICOM
fields moloney/dcmstack#39, but they want to
stick with just reporting the content of DICOM headers (without normalizing
scanner specific fields). Having said that I am planning to release a tool
that will output BIDS compatible .json file based on dcmstack output.

—
Reply to this email directly or view it on GitHub
https://github.com/neurolabusc/dcm2niix/issues/4#issuecomment-157416474.

[chrisgorgo]

Thank you @halfSpinDoctor - your input will be very much appreciated.

@neurolabusc BTW SPM also uses X, Y, Z terms to describe PhaseEncodingDirection (see http://www.fil.ion.ucl.ac.uk/spm/doc/manual.pdf#Chap:FieldMap)

[neurolabusc]

I suggest looking at the "3D IMAGE (VOLUME) ORIENTATION AND LOCATION IN SPACE" section of
http://nifti.nimh.nih.gov/pub/dist/src/niftilib/nifti1.h
to see the logic of decoupling world space (x,y,z) and world space (i,j,k). I am pretty sure that SPM's field map and fugue pre-date the NIfTI format. The DICOM use of rows/columns is here
http://dicomlookup.com/lookup.asp?sw=Tnumber&q=(0018,1312)
In any case, I think the code I have provided will allow you to validate, refine and maintain BIDS support for dcm2niix. Be aware I have not extensively tested my routines, they are just place holders for you. In addition, at the moment I only get slice timing data for Siemens, and therefore you will want to get some datasets for GE and Philips. I realize they are DTI instead of fMRI sequences, but you might want to look at the sample datasets here
https://www.nitrc.org/plugins/mwiki/index.php/dcm2nii:MainPage#Diffusion_Tensor_Imaging
to extract those values. If you do have datasets acquired to validate your tool, you may want to see if you can donate them to the Rosetta BIT project so that other developers will benefit.

[neurolabusc]

By the way, I am happy to merge any improvements you have. I have collaborated closely with Xiangrui Li, so his Matlab based dicm2nii and my C code share a lot of code. As far as I know:
a.) slice direction parameters for Philips data: Xiangrui's code extracts this, should easy to port dcm2niix.
b.) as far as I am aware, neither tool determines slice timing details for Philips. Note sure how to do this. We would probably want one of the groups with a Philips scanner to generate sample datasets to validate and test. I really do not know if this information is stored in a Philips dataset.
c.) I do think you would want to store the various Philips scaling values in BIDS. My software reports these. Where possible I convert these to NIfTI, but there are situations where this is not possible. (see PMC3998685)
d.) Last I checked slice timing information is not saved by GE, and in fact as I recall on some systems the stored order depends on how the image is planned at the time of acquisition rather than the sequence. Therefore, the identical sequence will generate DICOM image numbers differently if the operator first clicks the top or bottom of the screen. Therefore, we can determine the spatial position of a slice (based on spatial coordinates) but not the temporal order within a volume. This may have been fixed, but I do not have access to a GE system to verify this.

[halfSpinDoctor]

I can get you GE data if you would like.

I assume you just need a couple volumes acquired with the user clicking
superior and dragging out slices inferior, and then the other way round.

Did you want non-transversal slices also?

S

On Wednesday, 18 November 2015, Chris Rorden notifications@github.com
wrote:

By the way, I am happy to merge any improvements you have. I have
collaborated closely with Xiangrui Li, so his Matlab based dicm2nii and my
C code share a lot of code. As far as I know:
a.) slice direction parameters for Philips data: Xiangrui's code extracts
this, should easy to port dcm2niix.
b.) as far as I am aware, neither tool determines slice timing details for
Philips. Note sure how to do this. We would probably want one of the groups
with a Philips scanner to generate sample datasets to validate and test. I
really do not know if this information is stored in a Philips dataset.
c.) I do think you would want to store the various Philips scaling values
in BIDS. My software reports these. Where possible I convert these to
NIfTI, but there are situations where this is not possible. (see PMC3998685)
d.) Last I checked slice timing information is not saved by GE, and in
fact as I recall on some systems the stored order depends on how the image
is planned at the time of acquisition rather than the sequence. Therefore,
the identical sequence will generate DICOM image numbers differently if the
operator first clicks the top or bottom of the screen. Therefore, we can
determine the spatial position of a slice (based on spatial coordinates)
but not the temporal order within a volume. This may have been fixed, but I
do not have access to a GE system to verify this.

—
Reply to this email directly or view it on GitHub
https://github.com/neurolabusc/dcm2niix/issues/4#issuecomment-157747956.

---

Samuel A. Hurley, PhD
MRI Physics Support Scientist
FMRIB Centre, University of Oxford
Telephone: +44 (0) 1865 222733
http://users.fmrib.ox.ac.uk/~shurley/

[neurolabusc]

I am closing this issue. I have added experimental support for BIDS to the main branch. I would still like to get my hands on some GE data to validate this further.