Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FIX] Add OPTIONAL acq entity to channels.tsv, events.tsv to match electrophysiological acquisitions #677

Merged
merged 5 commits into from
Nov 30, 2020

Conversation

sappelhoff
Copy link
Member

follow up on #675 that @gpiantoni had to drop due to time constraints.

@sappelhoff
Copy link
Member Author

sappelhoff commented Nov 19, 2020

@tsalo I get the same issue as Gio upon calling python tools/bids_schema.py:

Traceback (most recent call last):
File "tools/bids_schema.py", line 467, in
_main()
File "tools/bids_schema.py", line 462, in _main
args.pop('func')
KeyError: 'func'

I think I have all the dependencies, so the error must be something else.

@tsalo
Copy link
Member

tsalo commented Nov 19, 2020

It's a matter of usage, I believe. Here is how you need to call it to update the entity table:

>>> python bids_schema.py entitytable ../src/schema/ \
>>> ../src/99-appendices/04-entity-table.md

cd tools/
python bids_schema.py entitytable ../src/schema/ ../src/99-appendices/04-entity-table.md
@sappelhoff
Copy link
Member Author

indeed 🤦‍♂️ thank you!

@effigies effigies changed the title [FIX] make acq entity optional for eeg, meg, ieeg, channels [FIX] Clarify optional acq entity for eeg, meg, ieeg, channels Nov 19, 2020
@effigies
Copy link
Collaborator

Is this new title accurate? My understanding is that we're clarifying, not adding.

@sappelhoff
Copy link
Member Author

sappelhoff commented Nov 19, 2020

Is this new title accurate? My understanding is that we're clarifying, not adding.

we are adding "acq" to "channels" for M(i)EEG, which wasn't present before, but which makes a lot of sense, if M(i)EEG JSONs can have that entity.

So it'd kind of ... a bugfix?

If you run a task (= 1 events.tsv) and record EEG with two amplifiers with potentially different settings (= 2 different *acq-<label>_eeg.<ext>), then the resulting channels files may HAVE to be different as well.

@effigies effigies changed the title [FIX] Clarify optional acq entity for eeg, meg, ieeg, channels [FIX] Add OPTIONAL acq entity to channels.tsv to match electrophysiological acquisitions Nov 19, 2020
@robertoostenveld
Copy link
Collaborator

I think this is a clarification, not an extension. The channels.tsv needs to match filename-wise with the eeg data itself and the eeg.json. If <acq>-label is part of the eeg dataset file name, it should also be part of the channels.tsv. Idem for example for runs, in case runs would have different filter settings (and hence channels.tsv would be run-specific).

@sappelhoff
Copy link
Member Author

I think this is a clarification, not an extension

to be clear: the questions whether this is a clarification or an extension is only important with regards to how we name the Pull Request.

Are you fine with the proposed changes other than the PR title, @robertoostenveld - or did you want to ask for additional changes?

@robertoostenveld
Copy link
Collaborator

I agree with all changes made in the PR, but wonder about a few that were not made. @gpiantoni might want to chime in

if two amplifiers were used, would there also be two events files? Since the start time of the data in the files is not identical, the event onsets in a single event file could only apply to one of them.

if two amplifiers were used, should there also be two electrode files? I mentioned this in https://neurostars.org/t/two-amplifiers-for-ieeg-recordings/17492/4, but technically I think that a single electrode file that contains both sets would suffice.

cd tools/
python bids_schema.py entitytable ../src/schema/ ../src/99-appendices/04-entity-table.md
@sappelhoff
Copy link
Member Author

agreed, an events file may be impacted as well. I have added that in the two most recent commits.

Electrode files already allowed for acq.

@sappelhoff sappelhoff changed the title [FIX] Add OPTIONAL acq entity to channels.tsv to match electrophysiological acquisitions [FIX] Add OPTIONAL acq entity to channels.tsv, events.tsv to match electrophysiological acquisitions Nov 20, 2020
@sappelhoff
Copy link
Member Author

@MaxvandenBoom @robertoostenveld @dorahermes could you please give this another look and an approving review if you agree?

Copy link
Collaborator

@robertoostenveld robertoostenveld left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good to me

@dorahermes
Copy link
Member

lgtm

@sappelhoff sappelhoff merged commit 72dc7d7 into bids-standard:master Nov 30, 2020
@sappelhoff sappelhoff deleted the optacq branch November 30, 2020 08:20
sappelhoff added a commit that referenced this pull request Dec 2, 2020
* [DOC] Auto-generate changelog entry for PR #677

* [ENH] BEP001 - New entities: inv & mt (#681)

* add entity inv

* add mt:

* add MT specific metadata

* Generate entity.md

* Update src/04-modality-specific-files/01-magnetic-resonance-imaging-data.md

Co-authored-by: Stefan Appelhoff <[email protected]>

* Update src/schema/entities.yaml

Co-authored-by: Stefan Appelhoff <[email protected]>

* table fix by @sappelhoff

* Conditional requirement

Co-authored-by: Stefan Appelhoff <[email protected]>

* [ENH] BEP001 - Entity-linked file collections (#688)

* Squashed commit of the following:

commit 8e1ac5f
Author: Stefan Appelhoff <[email protected]>
Date:   Fri Nov 27 11:09:24 2020 +0100

    fix links and latin

commit 27c9b41
Author: Stefan Appelhoff <[email protected]>
Date:   Fri Nov 27 10:03:51 2020 +0100

    remove latin

    Co-authored-by: Chris Markiewicz <[email protected]>

commit 7d87070
Author: Agah <[email protected]>
Date:   Mon Nov 23 08:46:47 2020 -0500

    Update src/99-appendices/10-file-collections.md

    Co-authored-by: Chris Markiewicz <[email protected]>

commit b137a9e
Author: Agah <[email protected]>
Date:   Mon Nov 23 08:45:19 2020 -0500

    Update src/02-common-principles.md

    Co-authored-by: Chris Markiewicz <[email protected]>

commit d7897b8
Author: Agah <[email protected]>
Date:   Mon Nov 23 08:44:40 2020 -0500

    Update src/02-common-principles.md

    Co-authored-by: Chris Markiewicz <[email protected]>

commit 09e9d32
Author: Agah <[email protected]>
Date:   Mon Nov 23 08:44:15 2020 -0500

    Commit suggestion

    Co-authored-by: Chris Markiewicz <[email protected]>

commit 009a9ab
Author: Agah Karakuzu <[email protected]>
Date:   Mon Nov 23 08:43:00 2020 -0500

    Add file-collecitons appendix to the TOC

commit 8f6c4df
Author: Agah Karakuzu <[email protected]>
Date:   Mon Nov 23 08:41:13 2020 -0500

    Address suggestion by @tsalo

commit 45210a9
Author: Agah Karakuzu <[email protected]>
Date:   Wed Nov 11 14:32:20 2020 -0500

    Wording

commit 66ba5a5
Author: Agah Karakuzu <[email protected]>
Date:   Wed Nov 11 14:31:40 2020 -0500

    RECOMMENDED --> MUST for adding an application def to the appdx

commit 5611fac
Author: Agah Karakuzu <[email protected]>
Date:   Wed Nov 11 14:29:21 2020 -0500

    Improve the appendix

commit 6d3bbea
Author: Agah Karakuzu <[email protected]>
Date:   Mon Nov 2 16:12:04 2020 -0500

    Address suggestions by @tsalo

commit bcb0223
Author: Agah Karakuzu <[email protected]>
Date:   Tue Oct 27 21:37:26 2020 -0400

    [ADD] Link to the appendix

commit 44e760f
Author: Agah Karakuzu <[email protected]>
Date:   Tue Oct 27 21:28:38 2020 -0400

    [ADD] Appendix - File collections

    - For parametrically linked file collections

commit 523b1c1
Author: Agah Karakuzu <[email protected]>
Date:   Tue Oct 27 21:27:52 2020 -0400

    [ADD] Parametrically linked file collections

    - Description

* ENH: Link to all BEP-001 entities

* STY: Escape asterisk, adjust table widths

* Add suggestions by @sappelhoff

* Update src/02-common-principles.md

Co-authored-by: Chris Markiewicz <[email protected]>

* Update src/02-common-principles.md

Co-authored-by: Chris Markiewicz <[email protected]>

* Update src/02-common-principles.md

Co-authored-by: Chris Markiewicz <[email protected]>

* Update src/99-appendices/10-file-collections.md

Co-authored-by: Chris Markiewicz <[email protected]>

* Update src/02-common-principles.md

Thank you @effigies!

Co-authored-by: Chris Markiewicz <[email protected]>

* Update src/02-common-principles.md

Co-authored-by: Chris Markiewicz <[email protected]>

Co-authored-by: Agah Karakuzu <[email protected]>

* [ENH] BEP001 - RepetitionTimeExcitation and RepetitionTimePreparation (#671)

* [ENH] Improve TR definitions

- Include RepetitionTimePreparation
- Include RepetitionTimeExcitation
- Add explanation on the former RepetitionTime field.

* Typo

* Update src/04-modality-specific-files/01-magnetic-resonance-imaging-data.md

Co-authored-by: Chris Markiewicz <[email protected]>

* ENH: Allow for variable RTE/P

* STY: Update table widths

* Update src/04-modality-specific-files/01-magnetic-resonance-imaging-data.md

* Update src/04-modality-specific-files/01-magnetic-resonance-imaging-data.md

* Change ref to DOI @Remi-Gau

* Update src/04-modality-specific-files/01-magnetic-resonance-imaging-data.md

Co-authored-by: Chris Markiewicz <[email protected]>

* Update src/04-modality-specific-files/01-magnetic-resonance-imaging-data.md

Co-authored-by: Chris Markiewicz <[email protected]>

Co-authored-by: Chris Markiewicz <[email protected]>
Co-authored-by: Chris Markiewicz <[email protected]>

* Update config.yml

* Update readthedocs.yml

Co-authored-by: bids-maintenance <[email protected]>
Co-authored-by: Agah <[email protected]>
Co-authored-by: Chris Markiewicz <[email protected]>
Co-authored-by: Chris Markiewicz <[email protected]>
tsalo added a commit to tsalo/bids-specification that referenced this pull request Dec 8, 2020
tsalo added a commit to tsalo/bids-specification that referenced this pull request Dec 8, 2020
sappelhoff added a commit that referenced this pull request Jan 15, 2021
* [ENH] Bep 005: Arterial Spin Labeling (#652)



Co-authored-by: Patricia Clement <[email protected]>
Co-authored-by: Stefan Appelhoff <[email protected]>
Co-authored-by: Chris Markiewicz <[email protected]>
Co-authored-by: Julia Guiomar Niso Galán <[email protected]>
Co-authored-by: Remi Gau <[email protected]>

* Update table errors

* correction for table issues 'common metadata for PCASL/PASL'

* correction latin phrases

* Corrections tables

* correction tables Cases

* added link pepolar

* deleted 'don't discriminate between types of labeling

based on comment Thijs Van Osch

* Moved MagneticFieldStrenght asl requirement to common

* removed scaling factor info based on comment gllmflndn

Removed:
all ancillary scaling factors should be taken into account in the conversion to BIDS, which is why BIDS does not provide a separate scaling factor field other than the NIfTI header.

* added MRAcquisitionType

added in the common - sequence specifics table
Reason: for ASL, defining 2D and 3D is required. This used to be added in the PulseSequenceType field, but this should stay a free text field. Therefore, the MRAcquisitionType field is added (based on DicomTag).

* removed PulseSequenceType from the ASL part

The 2D/3D information needed for ASL, is now moved to the new MRAcquisitionType field.

* added required common fields sentence

Some Common fields are required for ASL. To stress this out, we added a sentence for the *_as.json and the *_m0scan.json.

* adaptation requirements for m0scan.json

* adapted requirement level common fields for m0scan

* update sentence required common fields for asl.json

* Update common field EchoTime

* Removed EchoTime from ASL-table

* update FlipAngle in common RF and Contrasts

* remove FlipAngle from ASL fields

* update Dependency table ASL for MRAcquisitionType

* update SliceTiming in common fields

* remove SliceTiming from ASL tables

* added (FA) for FlipAngle

* update common RF and Contrast for table check

* LabelingPulseDuration and LabelingPulseInterval in ms instead of s

* required common metadatafields update

RepetitionTime or VolumeTiming to RepetitionTimePreparation

* update flipangle for timeseries versus file collections

* update echotime for timeseries

* update timing table (fence)

* update definition BackgroundSuppressionPulseTime

* removed common metadata fields from asl table

MagneticFieldStrenght, EchoTime, SliceTiming, VolumeTiming, RepetitionTime, FlipAngle, PulseSequenceType

* update scaling of asl and m0 files

- update scaling description for nifti files
- created separate scaling section (to highlight this important quantification factor), and M0 section
- moved scaling section and M0 section after aslcontext section

* update broken link in EchoTime (Timing Parameters)

* update broken link (2) EchoTime

* Update broken links FlipAngle

* LabelingType changed to ArterialSpinLabelingType

- changed in fields
- Changed in dependency table
- Dependency table: additionally removed repetitiontime/volumetiming parts

* Update dependency table based on new M0Type and M0Estimate fields

* Update new M0 field strategy

M0 field becomes M0Type, with addition field M0Estimate.
Also corrected IntendedFor description in m0 specific fields

* TEMPORARY: disable 'strict' docs build

this allows the build to proceed despite broken links

* changed fail_on_warning to true 

Intermediate solution to get BEP005 build untill the missing link is available

* see previous commit

* update bep005 with latest master + resolve conflict (#689)

* [DOC] Auto-generate changelog entry for PR #677

* [ENH] BEP001 - New entities: inv & mt (#681)

* add entity inv

* add mt:

* add MT specific metadata

* Generate entity.md

* Update src/04-modality-specific-files/01-magnetic-resonance-imaging-data.md

Co-authored-by: Stefan Appelhoff <[email protected]>

* Update src/schema/entities.yaml

Co-authored-by: Stefan Appelhoff <[email protected]>

* table fix by @sappelhoff

* Conditional requirement

Co-authored-by: Stefan Appelhoff <[email protected]>

* [ENH] BEP001 - Entity-linked file collections (#688)

* Squashed commit of the following:

commit 8e1ac5f
Author: Stefan Appelhoff <[email protected]>
Date:   Fri Nov 27 11:09:24 2020 +0100

    fix links and latin

commit 27c9b41
Author: Stefan Appelhoff <[email protected]>
Date:   Fri Nov 27 10:03:51 2020 +0100

    remove latin

    Co-authored-by: Chris Markiewicz <[email protected]>

commit 7d87070
Author: Agah <[email protected]>
Date:   Mon Nov 23 08:46:47 2020 -0500

    Update src/99-appendices/10-file-collections.md

    Co-authored-by: Chris Markiewicz <[email protected]>

commit b137a9e
Author: Agah <[email protected]>
Date:   Mon Nov 23 08:45:19 2020 -0500

    Update src/02-common-principles.md

    Co-authored-by: Chris Markiewicz <[email protected]>

commit d7897b8
Author: Agah <[email protected]>
Date:   Mon Nov 23 08:44:40 2020 -0500

    Update src/02-common-principles.md

    Co-authored-by: Chris Markiewicz <[email protected]>

commit 09e9d32
Author: Agah <[email protected]>
Date:   Mon Nov 23 08:44:15 2020 -0500

    Commit suggestion

    Co-authored-by: Chris Markiewicz <[email protected]>

commit 009a9ab
Author: Agah Karakuzu <[email protected]>
Date:   Mon Nov 23 08:43:00 2020 -0500

    Add file-collecitons appendix to the TOC

commit 8f6c4df
Author: Agah Karakuzu <[email protected]>
Date:   Mon Nov 23 08:41:13 2020 -0500

    Address suggestion by @tsalo

commit 45210a9
Author: Agah Karakuzu <[email protected]>
Date:   Wed Nov 11 14:32:20 2020 -0500

    Wording

commit 66ba5a5
Author: Agah Karakuzu <[email protected]>
Date:   Wed Nov 11 14:31:40 2020 -0500

    RECOMMENDED --> MUST for adding an application def to the appdx

commit 5611fac
Author: Agah Karakuzu <[email protected]>
Date:   Wed Nov 11 14:29:21 2020 -0500

    Improve the appendix

commit 6d3bbea
Author: Agah Karakuzu <[email protected]>
Date:   Mon Nov 2 16:12:04 2020 -0500

    Address suggestions by @tsalo

commit bcb0223
Author: Agah Karakuzu <[email protected]>
Date:   Tue Oct 27 21:37:26 2020 -0400

    [ADD] Link to the appendix

commit 44e760f
Author: Agah Karakuzu <[email protected]>
Date:   Tue Oct 27 21:28:38 2020 -0400

    [ADD] Appendix - File collections

    - For parametrically linked file collections

commit 523b1c1
Author: Agah Karakuzu <[email protected]>
Date:   Tue Oct 27 21:27:52 2020 -0400

    [ADD] Parametrically linked file collections

    - Description

* ENH: Link to all BEP-001 entities

* STY: Escape asterisk, adjust table widths

* Add suggestions by @sappelhoff

* Update src/02-common-principles.md

Co-authored-by: Chris Markiewicz <[email protected]>

* Update src/02-common-principles.md

Co-authored-by: Chris Markiewicz <[email protected]>

* Update src/02-common-principles.md

Co-authored-by: Chris Markiewicz <[email protected]>

* Update src/99-appendices/10-file-collections.md

Co-authored-by: Chris Markiewicz <[email protected]>

* Update src/02-common-principles.md

Thank you @effigies!

Co-authored-by: Chris Markiewicz <[email protected]>

* Update src/02-common-principles.md

Co-authored-by: Chris Markiewicz <[email protected]>

Co-authored-by: Agah Karakuzu <[email protected]>

* [ENH] BEP001 - RepetitionTimeExcitation and RepetitionTimePreparation (#671)

* [ENH] Improve TR definitions

- Include RepetitionTimePreparation
- Include RepetitionTimeExcitation
- Add explanation on the former RepetitionTime field.

* Typo

* Update src/04-modality-specific-files/01-magnetic-resonance-imaging-data.md

Co-authored-by: Chris Markiewicz <[email protected]>

* ENH: Allow for variable RTE/P

* STY: Update table widths

* Update src/04-modality-specific-files/01-magnetic-resonance-imaging-data.md

* Update src/04-modality-specific-files/01-magnetic-resonance-imaging-data.md

* Change ref to DOI @Remi-Gau

* Update src/04-modality-specific-files/01-magnetic-resonance-imaging-data.md

Co-authored-by: Chris Markiewicz <[email protected]>

* Update src/04-modality-specific-files/01-magnetic-resonance-imaging-data.md

Co-authored-by: Chris Markiewicz <[email protected]>

Co-authored-by: Chris Markiewicz <[email protected]>
Co-authored-by: Chris Markiewicz <[email protected]>

* Update config.yml

* Update readthedocs.yml

Co-authored-by: bids-maintenance <[email protected]>
Co-authored-by: Agah <[email protected]>
Co-authored-by: Chris Markiewicz <[email protected]>
Co-authored-by: Chris Markiewicz <[email protected]>

* update required common fields for m0scan.json

added EchoTime and FlipAngle in case LookLocker is true

* update dependency table

if PCASL : LabelingDuration is required
if PCASL: LabelingDuration should not be filled in

* update depedency table

removed 'PASL' LAbelingDuration should not be filled in

* Update src/04-modality-specific-files/01-magnetic-resonance-imaging-data.md

Co-authored-by: Chris Markiewicz <[email protected]>

* update description aslcontext

Co-authored-by: Chris Markiewicz <[email protected]>

* Update src/04-modality-specific-files/01-magnetic-resonance-imaging-data.md

Co-authored-by: Chris Markiewicz <[email protected]>

* Update src/04-modality-specific-files/01-magnetic-resonance-imaging-data.md

Co-authored-by: Chris Markiewicz <[email protected]>

* Update src/04-modality-specific-files/01-magnetic-resonance-imaging-data.md

Co-authored-by: Chris Markiewicz <[email protected]>

* Update src/04-modality-specific-files/01-magnetic-resonance-imaging-data.md

Co-authored-by: Chris Markiewicz <[email protected]>

* Update src/04-modality-specific-files/01-magnetic-resonance-imaging-data.md

Co-authored-by: Chris Markiewicz <[email protected]>

* update PCASLType and CASLType

Co-authored-by: Chris Markiewicz <[email protected]>

* updated required into REQUIRED

for asl.json and m0scan.json (common fields

* Added SummaryImages

Added images:
asl_pasl_boluscutoff_false.png
asl_pasl_boluscutoff_true_q2tips.png
asl_pasl_boluscutoff_true_quipssII.png
asl_pcasl_labeling_pulses.png
asl_pcasl_sequence.png

* create Appendix 11 for ASL

* Added asl_flowchart.png

* added subtitles for linking from main specification

* Added appendix link for control and label

* added appendix for asl sequences in general

* added appendix for (P)CASL

* added appendix for PASL

* added appendix for dependency table (flowcharts)

* MNT: Add ASL appendix to TOC

* TYPO: correct ASL appendix entry in tocwq

* fix md style

* Update src/04-modality-specific-files/01-magnetic-resonance-imaging-data.md

Co-authored-by: Chris Markiewicz <[email protected]>

* update table RF and Contrast

* update link Appendix XI to Appendix XI - ASL

* corrected link to appendix for PASL

* Update aslcontext cases

Case 1:
changed asl.json to asl.nii[.gz`]
Removed:  The optional deltam or cbf volumes should be stored and specified as derivative.

Case 2:
Removed: The optional cbf volumes should be stored and specified as derivative.

* Added aslcontext.tsv cases

this is moved from main spec bep005 to appendix XI - ASL

* Update aslcontext cases

- moved three cases to appendix XI - ASL
- added link to appendix XI
- added: Note that the raw images, including the m0scan, may also be used for quality control.

* corrected subtitles cases

* improve linebreaks in ASL appendix

* fix VascularCrushingVENC link

Based on comment of Stefan Appelhof, the Data Type for VascularChrushingVENC was fixed:
[number][] or \[array][] of [numbers][] 
to
[number][] or [array][] of [numbers][]

* added single column requirement for aslcontext.tsv

based on comment of Stefan Appelhof, the need for a single column was added in the introductory text for aslcontext.tsv, before the 3 cases.

* Changed VascularCrushingVENC to VascularCrushingVenc

* [SCHEMA] Add ASL to schema (#703)

* Add ASL to schema files and regenerate entity table.

* Fix style issue.

* Add fmap-format m0scan to schema.

* update M0Estimate fields

Added 'Referring to the M0 of blood'

* update LabelingDuration

Added additional information for LabelingDuration

* update labeling.jpg into *_asllabeling.jpg

* Change labeling to asllabeling. (#709)

* RF: Renumber ASL appendix to XII

Co-authored-by: Henk Mutsaerts <[email protected]>
Co-authored-by: Patricia Clement <[email protected]>
Co-authored-by: Chris Markiewicz <[email protected]>
Co-authored-by: Julia Guiomar Niso Galán <[email protected]>
Co-authored-by: Remi Gau <[email protected]>
Co-authored-by: bids-maintenance <[email protected]>
Co-authored-by: Agah <[email protected]>
Co-authored-by: Chris Markiewicz <[email protected]>
Co-authored-by: Taylor Salo <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants