Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[ENH] Encode mutually exclusive extensions in schema with sub-lists #1492

Open
wants to merge 27 commits into
base: master
Choose a base branch
from
Open

[ENH] Encode mutually exclusive extensions in schema with sub-lists #1492

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
3c72014
group extension
Remi-Gau May 18, 2023
13eabdd
add comment
Remi-Gau May 18, 2023
d3594e7
put all extension for data in list
Remi-Gau May 18, 2023
b0d5dd5
rename some funtions
Remi-Gau May 20, 2023
160b9c5
minor fixes
Remi-Gau May 20, 2023
1eb0f63
fx rendering
Remi-Gau May 20, 2023
bcf5ebc
attempt at fixing rules
Remi-Gau May 20, 2023
9708eb3
properly combine extensions
Remi-Gau May 20, 2023
e4b2922
add doc
Remi-Gau May 20, 2023
f28d3c2
import annotation for backward compatibility
Remi-Gau May 20, 2023
cd1da9d
add type hints
Remi-Gau May 20, 2023
0b05f28
Merge remote-tracking branch 'upstream/master' into one_extension
effigies May 26, 2023
341f7ae
RF: Fix extension list combination, remove some indirection
effigies May 29, 2023
d8e3ec6
update README
Remi-Gau May 29, 2023
e383905
TEST: Fix rendering tests
effigies May 31, 2023
a8bd1c3
RF: Convert extensions to lists-of-lists-of-strings
effigies May 30, 2023
586cd18
FIX: .mhd files are actually .raw.mhd
effigies May 31, 2023
d7e3f89
Merge remote-tracking branch 'upstream/master' into one_extension
effigies Jun 6, 2023
1fbb60b
MNT: Bump schema minor version to mark change to structure
effigies Jun 6, 2023
f693273
Merge branch 'master' into one_extension
effigies Jun 28, 2023
0d291c9
Merge branch 'master' into one_extension
effigies Jul 2, 2023
53c4fd3
Update src/schema/rules/files/raw/photo.yaml
Remi-Gau Jul 3, 2023
da67b35
remove digitizer
Remi-Gau Aug 10, 2023
6c092d8
fix some failing tests
Remi-Gau Aug 10, 2023
2ba8439
Merge branch 'master' into one_extension
effigies Aug 8, 2024
f297c30
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Aug 8, 2024
3958ac0
fix: Adapt to extensions being list[list[str]]
effigies Aug 8, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions src/appendices/meg-file-formats.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -329,8 +329,8 @@ A guide for using macros can be found at
"sub-control01_ses-001_task-rest_run-01_meg": "",
"sub-control01_ses-001_task-rest_run-01_meg.json": "",
"sub-control01_ses-001_task-rest_run-01_channels.tsv": "",
"sub-control01_ses-001_task-rest[_acq-<label>]_run-01_markers.<mrk,sqd>": "",
"sub-control01_ses-001_task-rest_run-01_meg.<con,sqd>": "",
"sub-control01_ses-001_task-rest[_acq-<label>]_run-01_markers.<mrk|sqd>": "",
"sub-control01_ses-001_task-rest_run-01_meg.<con|sqd>": "",
},
},
},
Expand Down
4 changes: 2 additions & 2 deletions src/metaschema.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -657,7 +657,7 @@
"items": { "pattern": "^[a-z]+$" }
},
"stem": { "type": "string" },
"extensions": { "type": "array", "items": { "type": "string" } }
"extensions": { "type": "array", "items": { "type": "array", "items": { "type": "string" } } }
},
"required": ["level", "stem", "extensions"],
"additionalProperties": false
Expand All @@ -674,7 +674,7 @@
"type": "array",
"items": { "pattern": "^[a-zA-Z0-9]+$" }
},
"extensions": { "type": "array", "items": { "type": "string" } },
"extensions": { "type": "array", "items": { "type": "array", "items": { "type": "string" } } },
"entities": {
"type": "object",
"patternProperties": {
Expand Down
5 changes: 1 addition & 4 deletions src/schema/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -654,10 +654,7 @@ README:
level: required
stem: README
extensions:
- ''
- .md
- .rst
- .txt
- ['', '.md', '.rst', '.txt']
```

Here, `README` and `README.md` are both valid, while only `dataset_description.json` is permitted.
Expand Down
2 changes: 1 addition & 1 deletion src/schema/objects/extensions.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -156,7 +156,7 @@ mefd:

Each recording consists of a `.mefd` directory.
mhd:
value: .mhd
value: .raw.mhd
display_name: ITAB Binary Header
description: |
Produced by ITAB-ARGOS153 systems. This file a binary header file, and is generated along with a
Expand Down
5 changes: 1 addition & 4 deletions src/schema/rules/files/common/core.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,7 @@ README:
level: recommended
stem: README
extensions:
- ''
- .md
- .rst
- .txt
- ['', .md, .rst, .txt]
CHANGES:
level: optional
path: CHANGES
Expand Down
12 changes: 4 additions & 8 deletions src/schema/rules/files/deriv/imaging.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -103,8 +103,7 @@ anat_parametric_discrete_segmentation:
density: optional
description: optional
extensions:
- .nii.gz
- .nii
- [.nii.gz, .nii]
- .json
- .tsv

Expand All @@ -119,8 +118,7 @@ anat_nonparametric_discrete_segmentation:
density: optional
description: optional
extensions:
- .nii.gz
- .nii
- [.nii.gz, .nii]
- .json
- .tsv

Expand Down Expand Up @@ -199,8 +197,7 @@ anat_parametic_discrete_surface:
suffixes:
- dseg
extensions:
- .label.gii
- .dlabel.nii
- [.label.gii, .dlabel.nii]
- .json
- .tsv
entities:
Expand All @@ -216,8 +213,7 @@ anat_nonparametic_discrete_surface:
suffixes:
- dseg
extensions:
- .label.gii
- .dlabel.nii
- [.label.gii, .dlabel.nii]
- .json
- .tsv
entities:
Expand Down
27 changes: 9 additions & 18 deletions src/schema/rules/files/raw/anat.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,7 @@ nonparametric:
- FLASH # deprecated
- PD # deprecated
extensions:
- .nii.gz
- .nii
- [.nii.gz, .nii]
- .json
datatypes:
- anat
Expand Down Expand Up @@ -50,8 +49,7 @@ parametric:
- S0map
- M0map
extensions:
- .nii.gz
- .nii
- [.nii.gz, .nii]
- .json
datatypes:
- anat
Expand All @@ -69,8 +67,7 @@ defacemask:
suffixes:
- defacemask
extensions:
- .nii.gz
- .nii
- [.nii.gz, .nii]
- .json
datatypes:
- anat
Expand All @@ -90,8 +87,7 @@ multiecho:
- MESE
- MEGRE
extensions:
- .nii.gz
- .nii
- [.nii.gz, .nii]
- .json
datatypes:
- anat
Expand All @@ -111,8 +107,7 @@ multiflip:
suffixes:
- VFA
extensions:
- .nii.gz
- .nii
- [.nii.gz, .nii]
- .json
datatypes:
- anat
Expand All @@ -133,8 +128,7 @@ multiinversion:
suffixes:
- IRT1
extensions:
- .nii.gz
- .nii
- [.nii.gz, .nii]
- .json
datatypes:
- anat
Expand All @@ -154,8 +148,7 @@ mp2rage:
suffixes:
- MP2RAGE
extensions:
- .nii.gz
- .nii
- [.nii.gz, .nii]
- .json
datatypes:
- anat
Expand All @@ -178,8 +171,7 @@ vfamt:
- MPM
- MTS
extensions:
- .nii.gz
- .nii
- [.nii.gz, .nii]
- .json
datatypes:
- anat
Expand All @@ -201,8 +193,7 @@ mtr:
suffixes:
- MTR
extensions:
- .nii.gz
- .nii
- [.nii.gz, .nii]
- .json
datatypes:
- anat
Expand Down
6 changes: 2 additions & 4 deletions src/schema/rules/files/raw/dwi.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,7 @@ dwi:
suffixes:
- dwi
extensions:
- .nii.gz
- .nii
- [.nii.gz, .nii]
- .json
- .bvec
- .bval
Expand All @@ -24,8 +23,7 @@ sbref:
suffixes:
- sbref
extensions:
- .nii.gz
- .nii
- [.nii.gz, .nii]
- .json
datatypes:
- dwi
Expand Down
7 changes: 3 additions & 4 deletions src/schema/rules/files/raw/eeg.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,12 @@ eeg:
- eeg
extensions:
- .json
- .edf
- [.bdf, .edf, .eeg, .set]
# .vhdr, .vmrk, .eeg must work as triplet
- .vhdr
- .vmrk
- .eeg
- .set
# .set file with an OPTIONAL .fdt
- .fdt
- .bdf
datatypes:
- eeg
entities:
Expand Down
21 changes: 7 additions & 14 deletions src/schema/rules/files/raw/fmap.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,7 @@ fieldmaps:
- magnitude
- fieldmap
extensions:
- .nii.gz
- .nii
- [.nii.gz, .nii]
- .json
datatypes:
- fmap
Expand Down Expand Up @@ -46,8 +45,7 @@ pepolar_m0scan:
suffixes:
- m0scan
extensions:
- .nii.gz
- .nii
- [.nii.gz, .nii]
- .json
datatypes:
- fmap
Expand All @@ -65,8 +63,7 @@ TB1DAM:
suffixes:
- TB1DAM
extensions:
- .nii.gz
- .nii
- [.nii.gz, .nii]
- .json
datatypes:
- fmap
Expand All @@ -86,8 +83,7 @@ TB1EPI:
suffixes:
- TB1EPI
extensions:
- .nii.gz
- .nii
- [.nii.gz, .nii]
- .json
datatypes:
- fmap
Expand All @@ -111,8 +107,7 @@ RFFieldMaps:
- TB1RFM
- RB1COR
extensions:
- .nii.gz
- .nii
- [.nii.gz, .nii]
- .json
datatypes:
- fmap
Expand All @@ -133,8 +128,7 @@ TB1SRGE:
suffixes:
- TB1SRGE
extensions:
- .nii.gz
- .nii
- [.nii.gz, .nii]
- .json
datatypes:
- fmap
Expand All @@ -156,8 +150,7 @@ parametric:
- TB1map
- RB1map
extensions:
- .nii.gz
- .nii
- [.nii.gz, .nii]
- .json
datatypes:
- fmap
Expand Down
6 changes: 2 additions & 4 deletions src/schema/rules/files/raw/func.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,7 @@ func:
- cbv
- sbref
extensions:
- .nii.gz
- .nii
- [.nii.gz, .nii]
- .json
datatypes:
- func
Expand Down Expand Up @@ -35,8 +34,7 @@ phase:
suffixes:
- phase # deprecated
extensions:
- .nii.gz
- .nii
- [.nii.gz, .nii]
- .json
datatypes:
- func
Expand Down
8 changes: 3 additions & 5 deletions src/schema/rules/files/raw/ieeg.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,13 @@ ieeg:
suffixes:
- ieeg
extensions:
- .mefd/
- [.nwb, .edf, .mefd/, .eeg, .set]
- .json
- .edf
# .vhdr, .vmrk, .eeg must work as triplet
- .vhdr
- .eeg
- .vmrk
- .set
# .set file with an OPTIONAL .fdt
- .fdt
- .nwb
datatypes:
- ieeg
entities:
Expand Down
24 changes: 9 additions & 15 deletions src/schema/rules/files/raw/meg.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,19 +3,15 @@ meg:
suffixes:
- meg
extensions:
- / # corresponds to BTi/4D data
- .ds/
- .json
- .fif
- .sqd
- .con
- .raw
- .ave
- .mrk
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.mrk seems to only exist for markers suffix

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, some context in this thread: #638 (comment) and in particular this reponse (i.e., why .mrk is only for the markers suffix, but .sqd can also be used for the brain data): #653 (comment)

- .kdf
- .mhd
effigies marked this conversation as resolved.
Show resolved Hide resolved
- .trg
# / corresponds to BTi/4D data
- [/, .ds/, .fif, .sqd, .con, .raw, .kdf]
- .ave # mentioned once in the appendix
# .raw and .raw.mhd work as a pair
- .raw.mhd
# .kdf with possible additional files .chn, .trg and _digitizer.txt (see other suffix group below)
- .chn
- .trg
datatypes:
- meg
entities:
Expand Down Expand Up @@ -61,8 +57,7 @@ headshape:
suffixes:
- headshape
extensions:
- .*
- .pos
- [.*, .pos]
datatypes:
- meg
entities:
Expand All @@ -74,8 +69,7 @@ markers:
suffixes:
- markers
extensions:
- .sqd
- .mrk
- [.sqd, .mrk]
datatypes:
- meg
entities:
Expand Down
6 changes: 1 addition & 5 deletions src/schema/rules/files/raw/micr.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,11 +20,7 @@ microscopy:
- SPIM
- XPCT
extensions:
- .ome.tif
- .ome.btf
- .ome.zarr/
- .png
- .tif
- [.ome.tif, .ome.btf, .ome.zarr/, .png, .tif]
- .json
datatypes:
- micr
Expand Down
Loading