From b6003a3801c13ed8919e3fa4bec7db336038e9fe Mon Sep 17 00:00:00 2001 From: Taylor Salo Date: Tue, 22 Oct 2024 10:59:16 -0400 Subject: [PATCH 01/13] Improve fMRIPrep-related documentation. --- docs/theoryofoperation.rst | 131 +++++++++++++++++++++---------------- docs/usage_rapidtide.rst | 20 +++--- 2 files changed, 82 insertions(+), 69 deletions(-) diff --git a/docs/theoryofoperation.rst b/docs/theoryofoperation.rst index ceda0f8f1..0bab71ffa 100644 --- a/docs/theoryofoperation.rst +++ b/docs/theoryofoperation.rst @@ -72,7 +72,7 @@ delayed blood), 3) is slow, since you're doing way more calculation than you need to, and 4) doesn't necessarily get you the best noise removal, since the systemic noise signal recorded outside the brain has its own characteristics and noise mechanisms that may make it diverge somewhat from what is actually -getting into the brain (although on the plus side, it is inarguably non-neuronal, +getting into the brain (although on the plus side, it is inarguably non-neuronal, so you don't have to have any arguments about slow neuronal waves). In contrast rapidtide (lets say it means Rapid Time Delay) is the newer faster, @@ -117,16 +117,17 @@ Each of these steps (and substeps) has nuances which will be discussed below. Generation of Masks """"""""""""""""""" -By default, rapidtide calculates masks dynamically at run time. There -are 5 masks used: 1) the global mean mask, which determines which voxels -are used to generate the initial global mean regressor, 2) The -correlation mask, which determines which voxels you actually calculate -rapidtide fits in (what you are describing here), 3) the refine mask, -which selects which voxels are used to generate a refined regressor for -the next fitting pass, 4) the offset mask, which determines which voxels are -used to estimate the "zero" time of the delay distribution, -and 5) the GLM mask, which determines which -voxels have the rapidtide regressors removed. +By default, rapidtide calculates masks dynamically at run time. +There are 5 masks used: +1) the global mean mask, which determines which voxels +are used to generate the initial global mean regressor; +2) the correlation mask, which determines which voxels you actually calculate +rapidtide fits in (what you are describing here); +3) the refine mean mask, which selects which voxels are used to generate a refined regressor for +the next fitting pass; +4) the offset mask, which determines which voxels are used to estimate the "zero" time of +the delay distribution; and +5) the GLM mask, which determines which voxels have the rapidtide regressors removed. Below is a description of how this works currently. NB: this is not how I THOUGHT is worked - until I just looked at the code just now. It @@ -138,55 +139,61 @@ The default behavior is to first calculate the correlation mask using nilearn.masking.compute_epi_mask with default values. This is a complicated function, which I'm using as a bit of a black box. Documentation for it is here: -https://nilearn.github.io/stable/modules/generated/nilearn.masking.compute_epi_mask.html#nilearn.masking.compute_epi_mask. +https://nilearn.github.io/stable/modules/generated/nilearn.masking.compute_epi_mask.html#nilearn.masking.compute_epi_mask. If you have standard, non-zero-mean fMRI data, it seems to work pretty well, but you -can specify your own mask using --corrmask NAME[:VALSPEC] (include any +can specify your own mask using ``--corrmask NAME[:VALSPEC]`` (include any non-zero voxels in the file NAME in the mask. If VALSPEC is provided, only include voxels with integral values specified by VALSPEC in the mask). VALSPEC is a comma separated list of integers (1,2,7,12) and/or integer ranges (2-7,12-15) so you can make masks of complicated combinations of -regions from an atlas. So for example --corrmask mymask.nii.gz:1,7-9,54 +regions from an atlas. So for example ``--corrmask mymask.nii.gz:1,7-9,54`` would include any voxels in mymask with values of 1, 7, 8, 9, or 54, -whereas --corrmask mymask.nii.gz would include any non-zero voxels in +whereas ``--corrmask mymask.nii.gz`` would include any non-zero voxels in mymask. **For the global mean mask:** -If --globalmeaninclude MASK[:VALSPEC] is specified, include all voxels -selected by MASK[:VALSPEC]. If it is not specified, include all voxels -in the mask. Then, if --globalmeanexclude MASK[:VALSPEC] is specified, -remove any voxels selected by MASK[:VALSPEC] from the mask. If it is -not specified, don't change the mask. +If ``--globalmeaninclude MASK[:VALSPEC]`` is specified, include all voxels selected by ``MASK[:VALSPEC]``. +If it is not specified, include all voxels in the mask. +Then, if ``--globalmeanexclude MASK[:VALSPEC]`` is specified, +remove any voxels selected by ``MASK[:VALSPEC]`` from the mask. +If it is not specified, don't change the mask. **For the refine mean mask:** -If --refineinclude MASK[:VALSPEC] is specified, include all voxels -selected by MASK[:VALSPEC]. If it is not specified, include all voxels -in the correlation mask mask. Then if --refineexclude MASK[:VALSPEC] is specified, -remove any voxels selected by MASK[:VALSPEC] from the mask. If it is -not specified, don't change the mask. Then multiply by corrmask, since -you can't use voxels where rapidtide was not run to do refinement. +If ``--refineinclude MASK[:VALSPEC]`` is specified, include all voxels selected by ``MASK[:VALSPEC]``. +If it is not specified, include all voxels in the correlation mask mask. +Then if ``--refineexclude MASK[:VALSPEC]`` is specified, +remove any voxels selected by ``MASK[:VALSPEC]`` from the mask. +If it is not specified, don't change the mask. +Then multiply by the correlation mask, +since you can't use voxels where rapidtide was not run to do refinement. **For the offset mask** -If --offsetinclude MASK[:VALSPEC] is specified, include all voxels -selected by MASK[:VALSPEC]. If it is not specified, include all voxels -in the correlation mask. Then if --offsetexclude MASK[:VALSPEC] is specified, -remove any voxels selected by MASK[:VALSPEC] from the mask. If it is -not specified, don't change the mask. Then multiply by corrmask, and use the voxels within -the mask to generate a histogram of delay values. Calculate the offset of the peak of the delay histogram, +If ``--offsetinclude MASK[:VALSPEC]`` is specified, include all voxels selected by ``MASK[:VALSPEC]``. +If it is not specified, include all voxels in the correlation mask. +Then if ``--offsetexclude MASK[:VALSPEC]`` is specified, +remove any voxels selected by`` MASK[:VALSPEC]`` from the mask. +If it is not specified, don't change the mask. +Then multiply by the correlation mask, +and use the voxels within the mask to generate a histogram of delay values. +Calculate the offset of the peak of the delay histogram, and subtract this value from all delay values within the correlation mask. **For the GLM mask:** -Include all voxels, unless you are calculating a CVR map, in which caserates other than the TR. Therefore -the first step in moving regressor processing is to resample the moving regressor estimate to match the (oversampled) -data sample rate. +Include all voxels, unless you are calculating a CVR map, in which case rates other than the TR. +Therefore the first step in moving regressor processing is to resample the moving regressor +estimate to match the (oversampled) data sample rate. -**Temporal filtering:** By default, all data and moving regressors are temporally bandpass filtered to 0.009-0.15Hz -(our standard definition of the LFO band). This can be overridden with ``--filterband`` and ``--filterfreqs`` command line -options. +**Temporal filtering:** +By default, all data and moving regressors are temporally bandpass filtered to 0.009-0.15 Hz +(our standard definition of the LFO band). +This can be overridden with ``--filterband`` and ``--filterfreqs`` command line options. -Depending on your data (including pathology), and what you want to accomplish, using the default correlation -mask is not ideal. For example, if a subject has obvious pathology, you may want to exclude these voxels -from being used to generate the initial global mean signal estimate, or from being used in refinement. +Depending on your data (including pathology), +and what you want to accomplish, using the default correlation mask is not ideal. +For example, if a subject has obvious pathology, +you may want to exclude these voxels from being used to generate the initial global mean signal estimate, +or from being used in refinement. Initial Moving Signal Estimation @@ -203,9 +210,9 @@ invalidates my assumption that the global mean is a good initial estimate of the One way to combat this is to limit the brain region that you get your initial regressor from, so that you are only sampling a single "pool" of delays. For example, you could use a gray matter mask for the global regresor estimation, since white matter has a smaller contribution from -the moving blood signal, and tends to get blood much later than gray matter anyway. Just add the option -``--globalmeaninclude graymask.nii.gz`` to your rapidtide command line. If you are using -fmriprep, you can get a gray matter mask using: +the moving blood signal, and tends to get blood much later than gray matter anyway. +Just add the option ``--globalmeaninclude graymask.nii.gz`` to your rapidtide command line. +If you are using fMRIPrep, you can get a gray matter mask using: :: @@ -216,27 +223,35 @@ fmriprep, you can get a gray matter mask using: -bin \ graymask +.. tip:: + It's often a good idea to use that gray matter mask for the + ``--refineinclude`` and ``--offsetinclude`` parameters as well. + If you want to be even more proactive, you could select a more focal brain region that you think has unperturbed circulation. For an Alzheimer's study that I am currently working on, we ended up starting only from blood in right and left cerebellar gray matter (freesurfer aparc+aseg regions 8 and 47) on the theory that if circulation in your cerebellum -is too messed up, you're dead, so would not be in the dataset. That made our delay estimates work a lot better. -So we used the freesurfer parcellations from fmriprep, transformed to standard space, to do that -preselection, using the option ``--globalmeaninclude standardspaceaparcasegfilename.nii.gz:8,47``. - -fmriprep does not provide a standard space aparc+aseg file - it's in T1 native space at 1mm resolution -(because that's the space freesurfer works in). Resampling to standard space is easy, BUT you must -remember to use NearestNeighbor -interpolation, or you'll get smeared, averaged boundaries between brain regions, which you REALLY don't want. -This command should get you a ``standardspaceaparcasegfilename.nii.gz`` (you need to have ANTs installed for this): +is too messed up, you're dead, so would not be in the dataset. +That made our delay estimates work a lot better. +So we used the freesurfer parcellations from fMRIPrep, transformed to standard space, +to do that preselection, +using the option ``--globalmeaninclude sub-XXX_space-MNI152NLin6Asym_res-2_desc-aparcaseg_dseg.nii.gz:8,47``. + +fMRIPrep does not provide a standard space aparc+aseg file - it's in T1 native space at 1mm resolution +(because that's the space freesurfer works in). +Resampling to standard space is easy, BUT you must remember to use NearestNeighbor interpolation (or GenericLabel if you're using antsApplyTransforms), +or you'll get smeared, averaged boundaries between brain regions, which you REALLY don't want. +This command should get you a standard-space aparc+aseg file named +``sub-XXX_space-MNI152NLin6Asym_res-2_desc-aparcaseg_dseg.nii.gz`` +(you need to have ANTs installed for this): :: antsApplyTransforms \ -d 3 \ -i BIDSHOME/derivatives/sub-XXX/anat/sub-XXX_desc-aparcaseg_dseg.nii.gz \ - -o BIDSHOME/derivatives/sub-XXX/anat/mymnispace_desc-aparcaseg_dseg.nii.gz \ + -o BIDSHOME/derivatives/sub-XXX/anat/sub-XXX_space-MNI152NLin6Asym_res-2_desc-aparcaseg_dseg.nii.gz \ -r BIDSHOME/derivatives/sub-XXX/anat/sub-XXX_space-MNI152NLin6Asym_res-2_desc-preproc_T1w.nii.gz \ - --interpolation NearestNeighbor \ + --interpolation GenericLabel \ --transform BIDSHOME/derivatives/sub-XXX/anat/sub-XXX_from-T1w_to-MNI152NLin6Asym_mode-image_xfm.h5 @@ -247,14 +262,14 @@ This includes the following operations: **Oversampling:** In order to simplify delay calculation, rapidtide performs all delay estimation operations on data with a sample rate of 2Hz or faster. Since most fMRI is recorded with a TR > 0.5s, this is achieved by -oversampling the data. The oversampling factor can be specified explicitly +oversampling the data. The oversampling factor can be specified explicitly (using the ``--oversampfac`` command line argument), but if it is not given, for data with a sample rate of less than 2Hz, all data and regressors are internally upsampled by the lowest integral factor that results in a sample rate >= 2Hz. -**Regressor resampling:** In the case where we are using the global mean signal -as the moving signal, the moving signal estimate and the fMRI data have +**Regressor resampling:** In the case where we are using the global mean signal +as the moving signal, the moving signal estimate and the fMRI data have the same sample rate, but if we use external recordings, such as NIRS or etCO2 timecourses, these will in general have sample rates other than the TR, and may start before and/or end after the fMRI acquisition. diff --git a/docs/usage_rapidtide.rst b/docs/usage_rapidtide.rst index da68ff33b..ebbb684e7 100644 --- a/docs/usage_rapidtide.rst +++ b/docs/usage_rapidtide.rst @@ -277,7 +277,7 @@ distortion correction and alignment steps done in the HCP distort the stripes, b average enough subjects though, they get washed out. **Spatial filtering** - I generally do NOT apply any spatial filtering -during preprocessing for a variety of reasons. fmriprep doesn't do it, so I feel validated in this choice. +during preprocessing for a variety of reasons. fMRIPrep doesn't do it, so I feel validated in this choice. You can always do it later, and rapidtide lets you do spatial smoothing for the purpose of estimating the delayed regressor using the ``--gausssigma`` parameter. This turns out to stabilize the fits for rapidtide and is usually a good thing, however you probably don't want it for other processing (but that's ok - see below). @@ -306,16 +306,14 @@ processing in standard space if you've done that alignment - either is fine, but there are typically far fewer voxels at native resolution, so processing will probably be faster. On the flip side, having everything in standard space makes it easier to combine runs and subjects. -**fmriprep** - If you do preprocessing in fmriprep, the easiest file to use for input to rapidtide would be either -``derivatives/fmriprep/sub-XXX/ses-XXX/func/XXX_desc-preproc_bold.nii.gz`` (native space) or -``derivatives/fmriprep/sub-XXX/ses-XXX/func/XXX_space-MNI152NLin6Asym_res-2_desc-preproc_bold.nii.gz`` -(standard space - replace ``MNI152NLin6aAsym_res-2`` with whatever space and resolution you used if not the FSL compatible -one). One caveat - unless this has changed recently, fmriprep does *not* store the transforms needed to go from -native BOLD space to standard space, so you'll have to come up with that yourself either by fishing the transform -out of the workdir, or redoing the alignment. That's a pretty strong argument for using the standard space. In addition, -if you do the analysis in standard space, it makes it easier to use freesurfer parcellations and gray/white/csf -segmentations that fmriprep provides for further tuning the rapidtide analysis. See the "Theory of Operation" section -for more on this subject. +**fMRIPrep** - If you do preprocessing in fMRIPrep, +the easiest file to use for input to rapidtide would be either +``derivatives/fMRIPrep/sub-XXX/ses-XXX/func/XXX_desc-preproc_bold.nii.gz`` (native space) or +``derivatives/fMRIPrep/sub-XXX/ses-XXX/func/XXX_space-MNI152NLin6Asym_res-2_desc-preproc_bold.nii.gz`` +(standard space - replace ``MNI152NLin6aAsym_res-2`` with whatever space and resolution you used if not the FSL compatible one). +If you do the analysis in standard space, it makes it easier to use freesurfer parcellations and gray/white/csf +segmentations that fMRIPrep provides for further tuning the rapidtide analysis. +See the "Theory of Operation" section for more on this subject. **AFNI** - Here's a case where you have to take some care - as I mentioned above, rapidtide assumes "FSL-like" data by default. The most important difference between AFNI and FSL preprocessing (assuming you've put your AFNI data into From 685d28fd2a72ddc76deb5af4c9ce045bc4e0d8c1 Mon Sep 17 00:00:00 2001 From: Taylor Salo Date: Tue, 22 Oct 2024 10:59:29 -0400 Subject: [PATCH 02/13] Update theoryofoperation.rst --- docs/theoryofoperation.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/theoryofoperation.rst b/docs/theoryofoperation.rst index 0bab71ffa..8840f3fd5 100644 --- a/docs/theoryofoperation.rst +++ b/docs/theoryofoperation.rst @@ -224,6 +224,7 @@ If you are using fMRIPrep, you can get a gray matter mask using: graymask .. tip:: + It's often a good idea to use that gray matter mask for the ``--refineinclude`` and ``--offsetinclude`` parameters as well. From aee991ef9e95f81f909b5a531707a726cf5b65d1 Mon Sep 17 00:00:00 2001 From: Taylor Salo Date: Tue, 22 Oct 2024 11:18:12 -0400 Subject: [PATCH 03/13] Describe how to use other params with fMRIPrep. --- docs/usage_rapidtide.rst | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/docs/usage_rapidtide.rst b/docs/usage_rapidtide.rst index ebbb684e7..4136e0703 100644 --- a/docs/usage_rapidtide.rst +++ b/docs/usage_rapidtide.rst @@ -308,13 +308,21 @@ having everything in standard space makes it easier to combine runs and subjects **fMRIPrep** - If you do preprocessing in fMRIPrep, the easiest file to use for input to rapidtide would be either -``derivatives/fMRIPrep/sub-XXX/ses-XXX/func/XXX_desc-preproc_bold.nii.gz`` (native space) or -``derivatives/fMRIPrep/sub-XXX/ses-XXX/func/XXX_space-MNI152NLin6Asym_res-2_desc-preproc_bold.nii.gz`` +``derivatives/fmriprep/sub-XXX/ses-XXX/func/XXX_desc-preproc_bold.nii.gz`` (native space) or +``derivatives/fmriprep/sub-XXX/ses-XXX/func/XXX_space-MNI152NLin6Asym_res-2_desc-preproc_bold.nii.gz`` (standard space - replace ``MNI152NLin6aAsym_res-2`` with whatever space and resolution you used if not the FSL compatible one). If you do the analysis in standard space, it makes it easier to use freesurfer parcellations and gray/white/csf segmentations that fMRIPrep provides for further tuning the rapidtide analysis. See the "Theory of Operation" section for more on this subject. +You can pass the confounds file from fMRIPrep +(``derivatives/fmriprep/sub-XXX/ses-XXX/func/XXX_desc-confounds_timeseries.tsv``) +directly to rapidtide as ``--motionfile``. +However, if you want to use the ``--confoundfile`` parameter, +you need to create a reduced version of the confounds file with only the columns you want to use for confound regression. + +You can also load the confounds file to identify non-steady-state volumes to use for the ``--numtozero`` parameter. + **AFNI** - Here's a case where you have to take some care - as I mentioned above, rapidtide assumes "FSL-like" data by default. The most important difference between AFNI and FSL preprocessing (assuming you've put your AFNI data into NIFTI format) is that AFNI removes the mean from the preprocessed fMRI data From cf96831ec3a4b46d68746aeff8cd50998a850e7d Mon Sep 17 00:00:00 2001 From: Taylor Salo Date: Tue, 22 Oct 2024 11:30:42 -0400 Subject: [PATCH 04/13] Fix intersphinx_mapping dict. --- docs/conf.py | 27 +++++++++++++++++++-------- docs/usage_rapidtide.rst | 5 +++++ 2 files changed, 24 insertions(+), 8 deletions(-) diff --git a/docs/conf.py b/docs/conf.py index 7580eac8c..9de9e781b 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -375,15 +375,26 @@ def setup(app): "https://github.com/bbfrederick/" "rapidtide/blob/{revision}/" "{package}/{path}#L{lineno}", ) -# Example configuration for intersphinx: refer to the Python standard library. +# ----------------------------------------------------------------------------- +# intersphinx +# ----------------------------------------------------------------------------- +_python_version_str = f"{sys.version_info.major}.{sys.version_info.minor}" +_python_doc_base = f"https://docs.python.org/{_python_version_str}" intersphinx_mapping = { - "http://docs.python.org/3.6": None, - "http://docs.scipy.org/doc/numpy": None, - "http://docs.scipy.org/doc/scipy/reference": None, - "http://matplotlib.org/": None, - "http://scikit-learn.org/stable": None, - "http://nipy.org/nibabel/": None, - "http://pandas.pydata.org/pandas-docs/stable/": None, + "python": (_python_doc_base, None), + "numpy": ("https://numpy.org/doc/stable/", None), + "scipy": ( + "https://docs.scipy.org/doc/scipy/reference", + (None, "./_intersphinx/scipy-objects.inv"), + ), + "matplotlib": ( + "https://matplotlib.org/stable/", + (None, "https://matplotlib.org/stable/objects.inv"), + ), + "pandas": ("https://pandas.pydata.org/pandas-docs/stable/", None), + "nibabel": ("https://nipy.org/nibabel/", None), + "nilearn": ("http://nilearn.github.io/stable/", None), + "scikit-learn": ("http://scikit-learn.org/stable", None), } sphinx_gallery_conf = { diff --git a/docs/usage_rapidtide.rst b/docs/usage_rapidtide.rst index 4136e0703..94ec63e4a 100644 --- a/docs/usage_rapidtide.rst +++ b/docs/usage_rapidtide.rst @@ -323,6 +323,11 @@ you need to create a reduced version of the confounds file with only the columns You can also load the confounds file to identify non-steady-state volumes to use for the ``--numtozero`` parameter. +fMRIPrep includes the TR in the output NIfTI files' headers, so you don't need to provide ``--datatstep``, +and it usually performs slice timing correction +(unless you don't have slice timing information in your BIDS dataset or choose ``--ignore slicetiming``), +so you don't need to use ``--slicetiming``. + **AFNI** - Here's a case where you have to take some care - as I mentioned above, rapidtide assumes "FSL-like" data by default. The most important difference between AFNI and FSL preprocessing (assuming you've put your AFNI data into NIFTI format) is that AFNI removes the mean from the preprocessed fMRI data From 274178fe235126a0bf1b251682f92db5b63ff9c3 Mon Sep 17 00:00:00 2001 From: Taylor Salo Date: Tue, 22 Oct 2024 11:42:36 -0400 Subject: [PATCH 05/13] Use myst_parser instead of archived m2r. --- docs/conf.py | 11 +---------- pyproject.toml | 4 ++-- setup.py | 2 +- setupbackup/pyproject.toml | 4 ++-- setupbackup/setup.py | 2 +- test.toml | 4 ++-- 6 files changed, 9 insertions(+), 18 deletions(-) diff --git a/docs/conf.py b/docs/conf.py index 9de9e781b..555e5a112 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -17,8 +17,6 @@ import sys from datetime import datetime -from m2r import MdInclude - # If extensions (or modules to document with autodoc) are in another directory, # add these directories to sys.path here. If the directory is relative to the # documentation root, use os.path.abspath to make it absolute, like shown here. @@ -74,6 +72,7 @@ "sphinx.ext.intersphinx", "sphinx.ext.autosummary", "sphinx.ext.doctest", + "myst_parser", "numpydoc", "sphinx.ext.ifconfig", "sphinx.ext.linkcode", @@ -208,14 +207,6 @@ # https://github.com/rtfd/sphinx_rtd_theme/issues/117 def setup(app): app.add_css_file("theme_overrides.css") - # Fix to https://github.com/sphinx-doc/sphinx/issues/7420 - # from https://github.com/life4/deal/commit/7f33cbc595ed31519cefdfaaf6f415dada5acd94 - # from m2r to make `mdinclude` work - app.add_config_value("no_underscore_emphasis", False, "env") - app.add_config_value("m2r_parse_relative_links", False, "env") - app.add_config_value("m2r_anonymous_references", False, "env") - app.add_config_value("m2r_disable_inline_math", False, "env") - app.add_directive("mdinclude", MdInclude) # Add any extra paths that contain custom files (such as robots.txt or diff --git a/pyproject.toml b/pyproject.toml index cb565d260..0c7b055d6 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -70,7 +70,7 @@ doc = [ 'sphinx_rtd_theme', 'sphinx-argparse', 'sphinx-gallery', - 'm2r', + 'myst_parser', 'numpydoc', 'recommonmark', 'mistune', @@ -82,7 +82,7 @@ doc = [ #calcicc = 'rapidtide.scripts.calcicc:main' [tool.setuptools] -include-package-data = true +include-package-data = true [tool.setuptools.packages.find] include = ['rapidtide', 'cloud'] diff --git a/setup.py b/setup.py index 802380cf2..a0f2213cd 100644 --- a/setup.py +++ b/setup.py @@ -272,7 +272,7 @@ "sphinx_rtd_theme", "sphinx-argparse", "sphinx-gallery", - "m2r", + "myst_parser", "numpydoc", "recommonmark", "mistune==0.8.4", diff --git a/setupbackup/pyproject.toml b/setupbackup/pyproject.toml index 822d3edfc..09b8ffb0b 100644 --- a/setupbackup/pyproject.toml +++ b/setupbackup/pyproject.toml @@ -65,7 +65,7 @@ doc = [ 'sphinx_rtd_theme', 'sphinx-argparse', 'sphinx-gallery', - 'm2r', + 'myst_parser', 'numpydoc', 'recommonmark', 'mistune', @@ -77,7 +77,7 @@ doc = [ #calcicc = 'rapidtide.scripts.calcicc:main' [tool.setuptools] -include-package-data = true +include-package-data = true [tool.setuptools.packages.find] include = ['rapidtide', 'cloud'] diff --git a/setupbackup/setup.py b/setupbackup/setup.py index e306bd46a..ae910efac 100644 --- a/setupbackup/setup.py +++ b/setupbackup/setup.py @@ -269,7 +269,7 @@ "sphinx_rtd_theme", "sphinx-argparse", "sphinx-gallery", - "m2r", + "myst_parser", "numpydoc", "recommonmark", "mistune==0.8.4", diff --git a/test.toml b/test.toml index 1ca23088c..289259414 100644 --- a/test.toml +++ b/test.toml @@ -67,7 +67,7 @@ doc = [ 'sphinx_rtd_theme', 'sphinx-argparse', 'sphinx-gallery', - 'm2r', + 'myst_parser', 'numpydoc', 'recommonmark', 'mistune', @@ -79,7 +79,7 @@ doc = [ #calcicc = 'rapidtide.scripts.calcicc:main' [tool.setuptools] -include-package-data = true +include-package-data = true [tool.setuptools.packages.find] include = ['rapidtide', 'cloud'] From a22456187c00a37a801a000630aefc221db89fb5 Mon Sep 17 00:00:00 2001 From: Taylor Salo Date: Tue, 22 Oct 2024 11:44:55 -0400 Subject: [PATCH 06/13] Directly include md file. --- docs/index.rst | 2 +- docs/whats_new.rst | 1 - 2 files changed, 1 insertion(+), 2 deletions(-) delete mode 100644 docs/whats_new.rst diff --git a/docs/index.rst b/docs/index.rst index b881903ad..bf1d30205 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -137,7 +137,7 @@ Contents :maxdepth: 2 :caption: History: - whats_new.rst + ../CHANGELOG.md Indices and tables diff --git a/docs/whats_new.rst b/docs/whats_new.rst deleted file mode 100644 index 6336d7666..000000000 --- a/docs/whats_new.rst +++ /dev/null @@ -1 +0,0 @@ -.. mdinclude:: ../CHANGELOG.md From 786709c2390cebbfad65f6159d74fb21c5a72ffd Mon Sep 17 00:00:00 2001 From: Taylor Salo Date: Tue, 22 Oct 2024 12:00:31 -0400 Subject: [PATCH 07/13] Okay try using recommonmark. --- docs/conf.py | 11 ++++++++--- pyproject.toml | 2 -- setup.py | 2 -- setupbackup/pyproject.toml | 2 -- setupbackup/setup.py | 2 -- test.toml | 2 -- 6 files changed, 8 insertions(+), 13 deletions(-) diff --git a/docs/conf.py b/docs/conf.py index 555e5a112..1d1489f76 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -17,6 +17,8 @@ import sys from datetime import datetime +from recommonmark.parser import CommonMarkParser + # If extensions (or modules to document with autodoc) are in another directory, # add these directories to sys.path here. If the directory is relative to the # documentation root, use os.path.abspath to make it absolute, like shown here. @@ -72,7 +74,6 @@ "sphinx.ext.intersphinx", "sphinx.ext.autosummary", "sphinx.ext.doctest", - "myst_parser", "numpydoc", "sphinx.ext.ifconfig", "sphinx.ext.linkcode", @@ -94,8 +95,12 @@ # The suffix(es) of source filenames. # You can specify multiple suffix as a list of string: -# source_suffix = ['.rst', '.md'] -source_suffix = ".rst" +source_parsers = { + ".md": CommonMarkParser, +} + +source_suffix = [".rst", ".md"] + # The encoding of source files. # source_encoding = 'utf-8-sig' diff --git a/pyproject.toml b/pyproject.toml index 0c7b055d6..1c56df4dc 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -70,10 +70,8 @@ doc = [ 'sphinx_rtd_theme', 'sphinx-argparse', 'sphinx-gallery', - 'myst_parser', 'numpydoc', 'recommonmark', - 'mistune', ] #[project.scripts] diff --git a/setup.py b/setup.py index a0f2213cd..7b7cebb4c 100644 --- a/setup.py +++ b/setup.py @@ -272,10 +272,8 @@ "sphinx_rtd_theme", "sphinx-argparse", "sphinx-gallery", - "myst_parser", "numpydoc", "recommonmark", - "mistune==0.8.4", ], "tests": [ "codecov", diff --git a/setupbackup/pyproject.toml b/setupbackup/pyproject.toml index 09b8ffb0b..cf3e4b6e2 100644 --- a/setupbackup/pyproject.toml +++ b/setupbackup/pyproject.toml @@ -65,10 +65,8 @@ doc = [ 'sphinx_rtd_theme', 'sphinx-argparse', 'sphinx-gallery', - 'myst_parser', 'numpydoc', 'recommonmark', - 'mistune', ] #[project.scripts] diff --git a/setupbackup/setup.py b/setupbackup/setup.py index ae910efac..a174cc7b3 100644 --- a/setupbackup/setup.py +++ b/setupbackup/setup.py @@ -269,10 +269,8 @@ "sphinx_rtd_theme", "sphinx-argparse", "sphinx-gallery", - "myst_parser", "numpydoc", "recommonmark", - "mistune==0.8.4", ], "tests": [ "codecov", diff --git a/test.toml b/test.toml index 289259414..f290980d9 100644 --- a/test.toml +++ b/test.toml @@ -67,10 +67,8 @@ doc = [ 'sphinx_rtd_theme', 'sphinx-argparse', 'sphinx-gallery', - 'myst_parser', 'numpydoc', 'recommonmark', - 'mistune', ] #[project.scripts] From 39d56160431ff8ec8ee1228d6e402e5ba93adb2a Mon Sep 17 00:00:00 2001 From: Taylor Salo Date: Tue, 22 Oct 2024 12:21:53 -0400 Subject: [PATCH 08/13] Hopefully these recs make sense. --- docs/theoryofoperation.rst | 22 +++++++++------------- docs/usage_rapidtide.rst | 23 ++++++++++++++++++++--- 2 files changed, 29 insertions(+), 16 deletions(-) diff --git a/docs/theoryofoperation.rst b/docs/theoryofoperation.rst index 8840f3fd5..d28b84ed1 100644 --- a/docs/theoryofoperation.rst +++ b/docs/theoryofoperation.rst @@ -211,22 +211,18 @@ One way to combat this is to limit the brain region that you get your initial re sampling a single "pool" of delays. For example, you could use a gray matter mask for the global regresor estimation, since white matter has a smaller contribution from the moving blood signal, and tends to get blood much later than gray matter anyway. -Just add the option ``--globalmeaninclude graymask.nii.gz`` to your rapidtide command line. -If you are using fMRIPrep, you can get a gray matter mask using: -:: - - fslmaths \ - BIDSHOME/derivatives/fmriprep/sub-XXX/anat/sub-YYY_space-MNI152NLin6Asym_res-2_label-GM_probseg.nii.gz \ - -s 3 \ - -thr 0.25 \ - -bin \ - graymask +Just add the option ``--graymattermask graymask.nii.gz`` to your rapidtide command line. +If you are using fMRIPrep, you can use the gray matter probabilistic map directly, +as rapidtide will threshold it automatically: -.. tip:: +.. bash:: - It's often a good idea to use that gray matter mask for the - ``--refineinclude`` and ``--offsetinclude`` parameters as well. + rapidtide \ + ... \ + --brainmask sub-XXX/anat/sub-XXX_space-MNI152NLin6Asym_res-2_desc-brain_mask.nii.gz \ + --graymattermask sub-XXX/anat/sub-XXX_space-MNI152NLin6Asym_res-2_desc-GM_probseg.nii.gz \ + --whitemattermask sub-XXX/anat/sub-XXX_space-MNI152NLin6Asym_res-2_desc-WM_probseg.nii.gz If you want to be even more proactive, you could select a more focal brain region that you think has unperturbed circulation. For an Alzheimer's study that I am currently working on, we ended up starting only from blood in right and diff --git a/docs/usage_rapidtide.rst b/docs/usage_rapidtide.rst index 94ec63e4a..6f7104c43 100644 --- a/docs/usage_rapidtide.rst +++ b/docs/usage_rapidtide.rst @@ -277,10 +277,12 @@ distortion correction and alignment steps done in the HCP distort the stripes, b average enough subjects though, they get washed out. **Spatial filtering** - I generally do NOT apply any spatial filtering -during preprocessing for a variety of reasons. fMRIPrep doesn't do it, so I feel validated in this choice. +during preprocessing for a variety of reasons. +fMRIPrep doesn't do it, so I feel validated in this choice. You can always do it later, and rapidtide lets you do spatial smoothing for the purpose of -estimating the delayed regressor using the ``--gausssigma`` parameter. This turns out to stabilize the fits for -rapidtide and is usually a good thing, however you probably don't want it for other processing (but that's ok - see below). +estimating the delayed regressor using the ``--gausssigma`` parameter. +This turns out to stabilize the fits for rapidtide and is usually a good thing, +however you probably don't want it for other processing (but that's ok - see below). **Temporal filtering** - Rapidtide does all it's own temporal filtering; highpass filtering at 0.01Hz, common in r esting state preprocessing, @@ -328,6 +330,21 @@ and it usually performs slice timing correction (unless you don't have slice timing information in your BIDS dataset or choose ``--ignore slicetiming``), so you don't need to use ``--slicetiming``. +For most non-clinical participants, +we recommend using the tissue type masks provided by fMRIPrep for many of the masks used in rapidtide. +For example: + +.. bash:: + + rapidtide \ + sub-XXX/func/sub-XXX_task-rest_space-MNI152NLin6Asym_res-2_desc-preproc_bold.nii.gz \ + /path/to/rapidtide/sub-XXX_task-rest_space-MNI152NLin6Asym_res-2 \ + --brainmask sub-XXX/anat/sub-XXX_space-MNI152NLin6Asym_res-2_desc-brain_mask.nii.gz \ + --graymattermask sub-XXX/anat/sub-XXX_space-MNI152NLin6Asym_res-2_desc-GM_probseg.nii.gz \ + --whitemattermask sub-XXX/anat/sub-XXX_space-MNI152NLin6Asym_res-2_desc-WM_probseg.nii.gz \ + --motionfile sub-XXX/func/sub-XXX_task-rest_desc-confounds_timeseries.tsv + + **AFNI** - Here's a case where you have to take some care - as I mentioned above, rapidtide assumes "FSL-like" data by default. The most important difference between AFNI and FSL preprocessing (assuming you've put your AFNI data into NIFTI format) is that AFNI removes the mean from the preprocessed fMRI data From f3716ab4bdab99b3cb8e96353dd8f3257db94d44 Mon Sep 17 00:00:00 2001 From: Taylor Salo Date: Tue, 22 Oct 2024 12:26:51 -0400 Subject: [PATCH 09/13] Okay make more recommendations. --- docs/theoryofoperation.rst | 35 ++++++++++++++++++++++------------- 1 file changed, 22 insertions(+), 13 deletions(-) diff --git a/docs/theoryofoperation.rst b/docs/theoryofoperation.rst index d28b84ed1..521d8e27f 100644 --- a/docs/theoryofoperation.rst +++ b/docs/theoryofoperation.rst @@ -129,10 +129,10 @@ the next fitting pass; the delay distribution; and 5) the GLM mask, which determines which voxels have the rapidtide regressors removed. -Below is a description of how this works currently. NB: this is not how -I THOUGHT is worked - until I just looked at the code just now. It -built up over time, and evolved into something that was not quite what I -designed. I'm going to fix it up, but this what it's doing as of 2.6.1, +Below is a description of how this works currently. +NB: this is not how I THOUGHT is worked - until I just looked at the code just now. +It built up over time, and evolved into something that was not quite what I designed. +I'm going to fix it up, but this what it's doing as of 2.6.1, which works most of the time, but may not be what you want. The default behavior is to first calculate the correlation mask using @@ -140,17 +140,26 @@ nilearn.masking.compute_epi_mask with default values. This is a complicated function, which I'm using as a bit of a black box. Documentation for it is here: https://nilearn.github.io/stable/modules/generated/nilearn.masking.compute_epi_mask.html#nilearn.masking.compute_epi_mask. -If you have -standard, non-zero-mean fMRI data, it seems to work pretty well, but you -can specify your own mask using ``--corrmask NAME[:VALSPEC]`` (include any -non-zero voxels in the file NAME in the mask. If VALSPEC is provided, only -include voxels with integral values specified by VALSPEC in the mask). +If you have standard, non-zero-mean fMRI data, it seems to work pretty well, +but you can specify your own mask using ``--corrmask NAME[:VALSPEC]`` +(include any non-zero voxels in the file NAME in the mask. +If VALSPEC is provided, only include voxels with integral values specified by VALSPEC in the mask). VALSPEC is a comma separated list of integers (1,2,7,12) and/or integer ranges (2-7,12-15) so you can make masks of complicated combinations of -regions from an atlas. So for example ``--corrmask mymask.nii.gz:1,7-9,54`` -would include any voxels in mymask with values of 1, 7, 8, 9, or 54, -whereas ``--corrmask mymask.nii.gz`` would include any non-zero voxels in -mymask. +regions from an atlas. +So for example ``--corrmask mymask.nii.gz:1,7-9,54`` would include any voxels in mymask +with values of 1, 7, 8, 9, or 54, +whereas ``--corrmask mymask.nii.gz`` would include any non-zero voxels in mymask. + +.. tip:: + + The ``--brainmask`` argument will automatically set all five of these masks, + but each can be overridden with the individual mask parameters. + +.. tip:: + + The ``--graymattermask`` argument will automatically set both the global mean mask and the offset mask, + but each can be overridden with the individual mask parameters. **For the global mean mask:** If ``--globalmeaninclude MASK[:VALSPEC]`` is specified, include all voxels selected by ``MASK[:VALSPEC]``. From 6faa4b45e0882a4a32251315a79363469cd075fa Mon Sep 17 00:00:00 2001 From: Taylor Salo Date: Tue, 22 Oct 2024 12:41:02 -0400 Subject: [PATCH 10/13] Fix code block. --- docs/conf.py | 4 +--- docs/theoryofoperation.rst | 19 ++++++++++--------- docs/usage_rapidtide.rst | 2 +- docs/whatsnew.rst | 2 ++ pyproject.toml | 2 +- setup.py | 2 +- setupbackup/pyproject.toml | 2 +- setupbackup/setup.py | 2 +- test.toml | 2 +- 9 files changed, 19 insertions(+), 18 deletions(-) create mode 100644 docs/whatsnew.rst diff --git a/docs/conf.py b/docs/conf.py index 1d1489f76..77b37ad49 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -17,8 +17,6 @@ import sys from datetime import datetime -from recommonmark.parser import CommonMarkParser - # If extensions (or modules to document with autodoc) are in another directory, # add these directories to sys.path here. If the directory is relative to the # documentation root, use os.path.abspath to make it absolute, like shown here. @@ -74,10 +72,10 @@ "sphinx.ext.intersphinx", "sphinx.ext.autosummary", "sphinx.ext.doctest", + "myst_parser", "numpydoc", "sphinx.ext.ifconfig", "sphinx.ext.linkcode", - "recommonmark", "sphinx_gallery.gen_gallery", ] diff --git a/docs/theoryofoperation.rst b/docs/theoryofoperation.rst index 521d8e27f..a7123acdc 100644 --- a/docs/theoryofoperation.rst +++ b/docs/theoryofoperation.rst @@ -119,15 +119,16 @@ Generation of Masks """"""""""""""""""" By default, rapidtide calculates masks dynamically at run time. There are 5 masks used: -1) the global mean mask, which determines which voxels -are used to generate the initial global mean regressor; -2) the correlation mask, which determines which voxels you actually calculate -rapidtide fits in (what you are describing here); -3) the refine mean mask, which selects which voxels are used to generate a refined regressor for -the next fitting pass; -4) the offset mask, which determines which voxels are used to estimate the "zero" time of -the delay distribution; and -5) the GLM mask, which determines which voxels have the rapidtide regressors removed. + +1. the global mean mask, which determines which voxels + are used to generate the initial global mean regressor; +2. the correlation mask, which determines which voxels you actually calculate + rapidtide fits in (what you are describing here); +3. the refine mean mask, which selects which voxels are used to generate a refined regressor for + the next fitting pass; +4. the offset mask, which determines which voxels are used to estimate the "zero" time of + the delay distribution; and +5. the GLM mask, which determines which voxels have the rapidtide regressors removed. Below is a description of how this works currently. NB: this is not how I THOUGHT is worked - until I just looked at the code just now. diff --git a/docs/usage_rapidtide.rst b/docs/usage_rapidtide.rst index 6f7104c43..6019b8474 100644 --- a/docs/usage_rapidtide.rst +++ b/docs/usage_rapidtide.rst @@ -334,7 +334,7 @@ For most non-clinical participants, we recommend using the tissue type masks provided by fMRIPrep for many of the masks used in rapidtide. For example: -.. bash:: +.. code-block:: bash rapidtide \ sub-XXX/func/sub-XXX_task-rest_space-MNI152NLin6Asym_res-2_desc-preproc_bold.nii.gz \ diff --git a/docs/whatsnew.rst b/docs/whatsnew.rst new file mode 100644 index 000000000..4afc7d3ff --- /dev/null +++ b/docs/whatsnew.rst @@ -0,0 +1,2 @@ +.. include:: ../CHANGELOG.md + :parser: myst_parser.sphinx_ diff --git a/pyproject.toml b/pyproject.toml index 1c56df4dc..b074c0e78 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -70,8 +70,8 @@ doc = [ 'sphinx_rtd_theme', 'sphinx-argparse', 'sphinx-gallery', + 'myst-parser', 'numpydoc', - 'recommonmark', ] #[project.scripts] diff --git a/setup.py b/setup.py index 7b7cebb4c..4899852ee 100644 --- a/setup.py +++ b/setup.py @@ -272,8 +272,8 @@ "sphinx_rtd_theme", "sphinx-argparse", "sphinx-gallery", + "myst-parser", "numpydoc", - "recommonmark", ], "tests": [ "codecov", diff --git a/setupbackup/pyproject.toml b/setupbackup/pyproject.toml index cf3e4b6e2..2d67dd523 100644 --- a/setupbackup/pyproject.toml +++ b/setupbackup/pyproject.toml @@ -65,8 +65,8 @@ doc = [ 'sphinx_rtd_theme', 'sphinx-argparse', 'sphinx-gallery', + 'myst-parser', 'numpydoc', - 'recommonmark', ] #[project.scripts] diff --git a/setupbackup/setup.py b/setupbackup/setup.py index a174cc7b3..ba2bff72f 100644 --- a/setupbackup/setup.py +++ b/setupbackup/setup.py @@ -269,8 +269,8 @@ "sphinx_rtd_theme", "sphinx-argparse", "sphinx-gallery", + "myst-parser", "numpydoc", - "recommonmark", ], "tests": [ "codecov", diff --git a/test.toml b/test.toml index f290980d9..83c187883 100644 --- a/test.toml +++ b/test.toml @@ -67,8 +67,8 @@ doc = [ 'sphinx_rtd_theme', 'sphinx-argparse', 'sphinx-gallery', + 'myst-parser', 'numpydoc', - 'recommonmark', ] #[project.scripts] From d6bff8c7ed8f9bdde1578c40364e28a4683963e0 Mon Sep 17 00:00:00 2001 From: Taylor Salo Date: Tue, 22 Oct 2024 12:41:18 -0400 Subject: [PATCH 11/13] Update theoryofoperation.rst --- docs/theoryofoperation.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/theoryofoperation.rst b/docs/theoryofoperation.rst index a7123acdc..961e28863 100644 --- a/docs/theoryofoperation.rst +++ b/docs/theoryofoperation.rst @@ -226,7 +226,7 @@ Just add the option ``--graymattermask graymask.nii.gz`` to your rapidtide comma If you are using fMRIPrep, you can use the gray matter probabilistic map directly, as rapidtide will threshold it automatically: -.. bash:: +.. code-block:: bash rapidtide \ ... \ From cc95867c4cf0f9a53040aee11b9f2161e2bc2c09 Mon Sep 17 00:00:00 2001 From: Taylor Salo Date: Tue, 22 Oct 2024 13:26:41 -0400 Subject: [PATCH 12/13] Fix thing. --- docs/conf.py | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/docs/conf.py b/docs/conf.py index 77b37ad49..c699d1f18 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -93,12 +93,7 @@ # The suffix(es) of source filenames. # You can specify multiple suffix as a list of string: -source_parsers = { - ".md": CommonMarkParser, -} - -source_suffix = [".rst", ".md"] - +source_suffix = [".rst"] # The encoding of source files. # source_encoding = 'utf-8-sig' From 4058e26c75b4426da3967f45dce8fcfbbea2adec Mon Sep 17 00:00:00 2001 From: Taylor Salo Date: Tue, 22 Oct 2024 13:58:52 -0400 Subject: [PATCH 13/13] Update index.rst --- docs/index.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/index.rst b/docs/index.rst index bf1d30205..db5c6628a 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -137,7 +137,7 @@ Contents :maxdepth: 2 :caption: History: - ../CHANGELOG.md + whatsnew.rst Indices and tables