Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ENH] New fNIRS example dataset: (non-)auto finger/foot tapping #323

Merged

Conversation

robertoostenveld
Copy link
Collaborator

this replaces #317 and follows up on this conversation

Copy link
Member

@sappelhoff sappelhoff left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can ignore the checks of the "stable" validator -- but the "master" checks should succeed.

There is an issue with the phenotype folder that relates to some points in this discussion: bids-standard/bids-specification#914

see CI check

There are also more errors that could be interesting to look into to either fix the validator, this example, or the spec itself 👍

Thanks!

@robertoostenveld
Copy link
Collaborator Author

With two small changes to the nirs branch of the bids-validator of @rob-luke this dataset now passes the validation. The checks will have to be rerun after rob-luke/bids-validator#1 is merged.

There was one issues that I resolved using the .bidsignore file: the phenotype/practicelogbook.tsv has multiple rows for each participant (since they practiced the same task multiple times), but the validator does not seem to like this.

@sappelhoff
Copy link
Member

There was one issues that I resolved using the .bidsignore file: the phenotype/practicelogbook.tsv has multiple rows for each participant (since they practiced the same task multiple times), but the validator does not seem to like this.

Why not reformat the dataframe to "wide" format, e.g.

id ses1 ses2 ses3
1 22 32 11
2 44 34 22

instead of

id ses val
1 1 22
1 2 32
1 3 11
2 1 44
2 2 34
2 3 22

@robertoostenveld
Copy link
Collaborator Author

There was one issues that I resolved using the .bidsignore file: the phenotype/practicelogbook.tsv has multiple rows for each participant (since they practiced the same task multiple times), but the validator does not seem to like this.

Why not reformat the dataframe to "wide" format, e.g.

id ses1 ses2 ses3
1 22 32 11
2 44 34 22

instead of

id ses val
1 1 22
1 2 32
1 3 11
2 1 44
2 2 34
2 3 22

Because of human readability reasons: It has 8 columns (i.e. 7 data columns), which would result in a wide format that is 1+5*7=36 columns wide. On top of that, one of the 8 columns (the last) is "notes" in a free-text format that can contains whole sentences (or n/a when not specified).

An alternative solution that we considered (but also decided against) is to split it over 5 files. That also negatively affects the human readability, as you would have to scan over 5 files to see how performance (and subjective experience) changes over the 5 training days.

Since this dataset is not meant as a phenotype example, and since the phenotype validation is apparently not optimal yet, I consider it fine if the phenotype data in this dataset is treated as exception. I could also leave it out, but then the test dataset would diverge from the actual one.

@rob-luke
Copy link
Member

With two small changes to the nirs branch of the bids-validator of @rob-luke this dataset now passes the validation

Thank you Robert for this PR. It is now merged. Can someone please kick the CI to see if the master now goes green.

@rob-luke
Copy link
Member

This is a wonderful example dataset. I learnt lots from looking through it. It provides a great compliment to #305, beyond being a different device it has many differences to 305 including:

  • this dataset has x/y/z and mine has template/x/y/z,
  • this one has more complex optode names
  • this one has different nominal and actual wavelength
  • this dataset provides the data in optical density rather than cw amplitude
  • this one short channels specified
  • much more detailed metadata and descriptions
  • etc

Thanks for adding this example. It all looks great to me

@sappelhoff
Copy link
Member

Thank you Robert for this PR. It is now merged. Can someone please kick the CI to see if the master now goes green.

done, and it comes back green :-)

@robertoostenveld
Copy link
Collaborator Author

This is a wonderful example dataset. ...

That compliment goes to @helenacockx

@sappelhoff sappelhoff added this to the 1.8.0 milestone Jun 22, 2022
@helenacockx
Copy link

Thank you, Robert, for the adaptations to make it pass through the validator!
I am currently traveling through the USA and Canada to visit some labs here, so I don't have much time to look closely at it, but I will do it once I find the time!

@sappelhoff sappelhoff changed the base branch from master to bep030_nirs July 12, 2022 10:28
@helenacockx
Copy link

The updates from Robert look excellent! I have nothing more to add.

robertoostenveld and others added 7 commits September 1, 2022 12:45
 - fixed AnatomicalLandmarkCoordinateSy_t_em
 - fixed NIRSCoordinateUnit_e_s
 - removed txt from CHANGES
 - added stimulus MATLAB m-files to bidsignore
 - added sub-95 with n/a to practicelogbook.tsv
 - use n/a for empty cells in practicelogbook.tsv
 - added practicelogbook.tsv to bidsignore - HACK
 - removed sub-77 json files from sub-75/nirs directory
 - replaced 0-byte (empty) snirf files with 1-byte (still empty) snirf files - HACK
 - use forward slashes in scans.tsv
 - inserted "type" column in all optodes.tsv files
 - reordered the columns of channels.tsv
@sappelhoff sappelhoff changed the title added fNIRS example dataset [ENH] New fNIRS example dataset: (non-)auto finger/foot tapping Sep 1, 2022
@sappelhoff sappelhoff merged commit fa58fd2 into bids-standard:bep030_nirs Sep 1, 2022
@sappelhoff
Copy link
Member

Thanks @robertoostenveld

sappelhoff added a commit that referenced this pull request Sep 1, 2022
* fNIRS example dataset (BEP030) (#305)

* Add fNIRS example dataset

* Use nirs branch for validator

* Update fnirs_tapping/dataset_description.json

Co-authored-by: Stefan Appelhoff <stefan.appelhoff@mailbox.org>

* Fix spacing in name

* revert link change validator

Co-authored-by: Stefan Appelhoff <stefan.appelhoff@mailbox.org>

* [ENH] New fNIRS example dataset: (non-)auto finger/foot tapping (#323)

* added another example dataset for fNIRS, see #317

* use the nirs branch of the validator from rob-luke/bids-validator

* replaced "large" PDF file with a zero-byte version

* updated fNIRRS example dataset to pass the validator

 - fixed AnatomicalLandmarkCoordinateSy_t_em
 - fixed NIRSCoordinateUnit_e_s
 - removed txt from CHANGES
 - added stimulus MATLAB m-files to bidsignore
 - added sub-95 with n/a to practicelogbook.tsv
 - use n/a for empty cells in practicelogbook.tsv
 - added practicelogbook.tsv to bidsignore - HACK
 - removed sub-77 json files from sub-75/nirs directory
 - replaced 0-byte (empty) snirf files with 1-byte (still empty) snirf files - HACK
 - use forward slashes in scans.tsv
 - inserted "type" column in all optodes.tsv files
 - reordered the columns of channels.tsv

* renamed stim to stimuli

* added SourceType back to a JSON file where I accidentally deleted it

* revert validator branch change, add entry to README

Co-authored-by: Stefan Appelhoff <stefan.appelhoff@mailbox.org>

Co-authored-by: Rob Luke <code@robertluke.net>
Co-authored-by: Robert Oostenveld <r.oostenveld@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants