Validation error with '.sqd' MEG files

[JWinawer]

We get errors when validating a BIDS project containing MEG files with the extension '.sqd'. We understood from the BIDS specificiation that sqd was allowable. The error is this:

Error 1: [Code 1] NOT_INCLUDED

Files with such naming scheme are not part of BIDS specification. This error is most commonly caused by typos in file names that make them not BIDS compatible. Please consult the specification and make sure your files are named correctly. If this is not a file naming issue (for example when including files not yet covered by the BIDS specification) you should include a ".bidsignore" file in your dataset (see https://github.com/bids-standard/bids-validator#bidsignore for details). Please note that derived (processed) data should be placed in /derivatives folder and source data (such as DICOMS or behavioural logs in proprietary formats) should be placed in the /sourcedata folder.

sub-ny705_ses-nyumeg01_task-boldhand_run-1_markers.sqd
Location:
motor/sub-ny705/ses-nyumeg01/meg/sub-ny705_ses-nyumeg01_task-boldhand_run-1_markers.sqd

Reason:
Files with such naming scheme are not part of BIDS specification. This error is most commonly caused by typos in file names that make them not BIDS compatible. Please consult the specification and make sure your files are named correctly. If this is not a file naming issue (for example when including files not yet covered by the BIDS specification) you should include a ".bidsignore" file in your dataset (see https://github.com/bids-standard/bids-validator#bidsignore for details). Please note that derived (processed) data should be placed in /derivatives folder and source data (such as DICOMS or behavioural logs in proprietary formats) should be placed in the /sourcedata folder.

Evidence:
sub-ny705_ses-nyumeg01_task-boldhand_run-1_markers.sqd

Any tips appreciated.

Thanks
Jon

[effigies]

Seems like a bug. SQD should be recognized by this regular expression (though it is a pain to try to read):

https://github.com/bids-standard/bids-validator/blob/392af3ca8c6a172c5e87b210c0b56ec45989d92d/bids-validator/bids_validator/rules/file_level_rules.json#L204-L220

Is there any chance that there's something more fundamental? For instance, where does motor/ come into this? Is that the dataset root? Could you share the tree of your whole dataset?

[irisgroen]

Hi Chris, thanks for your reply - Jon and I work on this project together. Here's a screenshot of our 'tree', with 'motor' being the top-level directory that we input into the bids-validator. The reason we think this is a bug having to do with the .sqd extension is that our subjects also have data from other measurement modalities (e.g. fMRI , in the session 'nyu3t01' in the highlighted example) with the same naming scheme and tree structure and we don't get errors for any of those files, or for the .tsv and .json files with the same names; only for the _markers.sqd and _meg.sqd for all the subjects in the dataset.

In case it's helpful, I've also attached the validator error file here (as you can see there are two more errors but we know why those are there and how to fix them).
motor_errors (1).txt

Could it be that the validator is not recognizing this session as meg somehow?

[effigies]

The file tree looks reasonable, which means, unfortunately, it's probably a regex problem.

[teonbrooks]

I think I may have found the error here:
https://github.com/bids-standard/bids-validator/blob/392af3ca8c6a172c5e87b210c0b56ec45989d92d/bids-validator/bids_validator/rules/file_level_rules.json#L205

specifically here:
_meg(@@@_meg_type_@@@\\/(.(?!\\.(sqd|con|fif|raw|raw\\.mhd|trg|kdf|chn)$))*|\\/(.(?!\\.(sqd|con|fif|raw|raw\\.mhd|trg|kdf|chn)$))*)|(@@@_meg_ext_@@@))$

why is there intervening content between _meg(@@@_meg_type_@@@ and (@@@_meg_ext_@@@)? I think it might trying a parsing error. I would expect that this line looked more below:

https://github.com/bids-standard/bids-validator/blob/392af3ca8c6a172c5e87b210c0b56ec45989d92d/bids-validator/bids_validator/rules/file_level_rules.json#L160
or
https://github.com/bids-standard/bids-validator/blob/392af3ca8c6a172c5e87b210c0b56ec45989d92d/bids-validator/bids_validator/rules/file_level_rules.json#L178

[effigies]

Seems reasonable to me... Want to make a PR?

[teonbrooks]

why is there intervening content between _meg(@@@_meg_type_@@@ and (@@@_meg_ext_@@@)?

i just understood what it is for, it is for the BTi data that have files with extensions: https://bids-specification.readthedocs.io/en/stable/99-appendices/06-meg-file-formats.html#bti4d-neuroimaging

[irisgroen]

Hi all, we've tested the validator again (using the online browser version) on our MEG data and we no longer get the .sqd validation error. I'm not sure if it was solved by #850 or not, but regardless I think this means this issue can be closed!

[effigies]

This might be an issue where the web version of the validator hadn't updated since June...

We got it updating again last week.

[teonbrooks]

I'm not sure if it was solved by #850 or not

this one didn't, I was attempting to fix it but I just kept breaking things on my branch hehe. glad to hear it was resolved. i think it could have been a dated version. I know at one point, we had an error in the specification where we were requiring an additional enclosing folder on KIT data that was not necessary (bids-standard/bids-specification#19).