[ENH] BEP031 - New columns to participants.tsv file #816

mariehbourget · 2021-06-04T19:51:50Z

Dear BIDS community,

Context

As part of the development of the Microscopy BEP (BEP031 @mariehbourget, @jcohenadad) and Animal Ephys BEP (BEP032 @SylvainTakerkart, @JuliaSprenger), the “sample” entity was introduced along with a samples.tsv file describing properties of the samples in PR #812.

To better describe animal participants, new properties of the subjects also need to be included in participants.tsv.

Contribution

The purpose of this PR is to add new recommended column (species) and optional columns (strain, strain_rrid and diagnosis) to the participants.tsv file.

See issue #779 for related discussions on this topic.

src/03-modality-agnostic-files.md

satra · 2021-06-04T22:25:02Z

src/03-modality-agnostic-files.md

-and `handedness`. We RECOMMEND to make use of these columns, and
-in case that you do use them, we RECOMMEND to use the following values
-for them:
+When different from `homo sapiens`, `participants.tsv` SHOULD include a `species`


shouldn't this be MUST, given that all of BIDS assumes humans at this point.

Suggested change

When different from `homo sapiens`, `participants.tsv` SHOULD include a `species`

When different from `homo sapiens`, `participants.tsv` MUST include a `species`

I don't know how we would validate a MUST, here. Also, there are rodent datasets that may not have this column in this form at this point, so we would be breaking backwards compatibility if we could validate. What about:

The RECOMMENDED `species` column MUST be a binomial species name from the [NCBI Taxonomy](https://www.ncbi.nlm.nih.gov/Taxonomy/Browser/wwwtax.cgi). For backwards compatibility, if `species` is absent, the participant is assumed to be `homo sapiens`.

Also, REQUIRE-ing a species name from NCBI Taxonomy feels like it's going to be difficult to validate, as we will need to either query the database or maintain a list of accepted names, updating the validator as new use cases arise... Is there a validation plan?

Thank you @effigies for the suggestion, I think assuming homo sapiens if the column is omitted is a strong incentive without breaking backward compatibility. I would be in favor of that.

However, I had not thought about the validation, querying the database seems like the best option to not have to maintain an up-to-date list in the validator but it may be difficult to implement. Are there similar requirements elsewhere in the spec? Would the alternative of “SHOULD” or “strongly RECOMMENDED” be advisable?

Also, thinking about it, I think I should add examples other than homo sapiens like mus musculus and rattus norvegicus in the description.

@effigies - regarding validation, i will raise the same issue here as the other post. i'm not sure we actually validate values for example for sex or anything that could have levels or enumerations.

I don't know how we would validate a MUST

while one could at least detect presence, i agree that keeping with the current perspective of the participants.tsv being a recommended file, we can keep things recommended instead of required.

species does get a little complicated, especially for animals, as you start going into species + genotype notions. here is our generic participant at a timepoint model in dandi: https://github.com/dandi/dandischema/blob/master/dandischema/models.py#L642 (technically all of those properties could come into play, with some being more important for animal studies).

@effigies, I modified the species description as you suggested and added examples in 8b41137.
For validation purposes, I kept both the column and the taxonomy as RECOMMENDED and not REQUIRED.
Let me know what you think.

satra · 2021-06-04T22:28:31Z

src/03-modality-agnostic-files.md

+-   `strain_rrid`: research resource identifier ([RRID](https://scicrunch.org/resources/Organisms/search))
+    of the strain of the species
+
+-   `diagnosis`: string value describing the diagnosis of the participant.


i don't know if this has to be a string value. in many datasets on openneuro diagnosis/dx is present and can be an enumerated type. also, this is one place, where one can have multiple designations depending on the study. we should allow for some notion of that, or simply remove diagnosis from this file.

The aim of this PR is to add new columns to describe animal properties. In that context, I agree that diagnosis may be out of scope.

For context, I added the columns following this discussion #779 (comment), #779 (comment) and #779 (comment) because we also introduced pathology in samples.tsv.

Following your suggestion, I removed the diagnosis column in 7146144 as being out of scope for this PR (animal properties).

effigies · 2021-06-06T16:06:16Z

src/03-modality-agnostic-files.md

@@ -197,6 +201,15 @@ for them:
    -   for "ambidextrous", use one of these values: `ambidextrous`, `a`, `A`,
        `AMBIDEXTROUS`, `Ambidextrous`

+-   `strain`: string value indicating the strain of the species


Examples for each of these would be useful.

Just to clarify, do you mean example directly in the description, like above for handedness with ambidextrous, a, A, etc, or an example of particpants.tsv for an animal.

I did not change the example of participants.tsv below which is an example for human. I thought having complete examples for both human and animal would maybe be too much.

perhaps a few examples from: https://www.jax.org/jax-mice-and-services/find-and-order-jax-mice/most-popular-jax-mice-strains

Examples added in b451671.

sappelhoff

I have a more general and perhaps controversial comment, that I'd like to hear opinions on.

Up until now BIDS has been mostly about humans and their brain+behavior data. I assume that >80% of users will continue to come to BIDS from this angle.

Adding coverage for animal ephys, and microscopy into BIDS is great, and I fully support it. However is it a good idea to introduce very specific concepts (such as "strain") that are not used in 80% of BIDS use-cases* into a very general section that discusses things like age, sex, and handedness (which are ubiquitous)? I feel like this would be better discussed in a dedicated, separate section of the spec (animal ephys / microscopy respectively), as it avoids clutter and potential confusion / annoyance with the spec.

*(this is my assumption/prediction, feel free to challenge).

rob-luke · 2021-07-16T10:21:28Z

src/03-modality-agnostic-files.md

+[NCBI Taxonomy](https://www.ncbi.nlm.nih.gov/Taxonomy/Browser/wwwtax.cgi).
+
+Commonly used *optional* columns in `participants.tsv` files are `age`, `sex`,
+`handedness`, `strain`, `strain_rrid` and `diagnosis`. We RECOMMEND to make use


Could group be added to this list as its used in the example below?

Good question, I'm not sure we should because group is also used below to illustrate how to describe an additional column that is not part of the optional ones in the participants.json example. From my understanding, group may have different meanings depending on the study and it would be hard to define it with a general example

mariehbourget · 2021-07-16T15:48:29Z

Adding coverage for animal ephys, and microscopy into BIDS is great, and I fully support it. However is it a good idea to introduce very specific concepts (such as "strain") that are not used in 80% of BIDS use-cases into a very general section that discusses things like age, sex, and handedness (which are ubiquitous)? I feel like this would be better discussed in a dedicated, separate section of the spec (animal ephys / microscopy respectively), as it avoids clutter and potential confusion / annoyance with the spec.

Thanks @sappelhoff for your feedback!
I changed the wording to make it clearer that it applies to species other than human and added an example for strain and strain_rrid, so it may create less confusion in that way.

The main goal of this PR is to promote species as a recommended field in participants.tsv to introduce animal data in the spec, as discussed previously with BEP032 and with the steering group.

We foresee that the BIDS-specification for microscopy will be adopted by many users, some have already started converting their datasets or working on compatible acquisition workflows, but I would not be able to quantify such adoption at this point.
So for microscopy and animal ephys, it is likely that strain and strain_rrid will apply for many users, but I agree that it would not be the case for 80% of BIDS at large.

I'm not against moving those field into the dedicated microscopy and animal ephys sections if we want to keep the general section more general. The main drawback being duplication in the specification.

mariehbourget · 2021-08-31T14:15:55Z

Hi @effigies and @sappelhoff,
I’m currently working on the microscopy .md file for the main BEP031 PR. I would like to know your thoughts about the optional columns strain and strain_rrid, following the latest changes and comment above.

I can move them to the microscopy part if it makes more sense to have them per modalities rather than in the “general” participants file.

In that case, this PR would only add the species columns to participants.
Thank you!

sappelhoff · 2021-08-31T15:28:38Z

tagging @Remi-Gau to chime in instead of me :-)

Remi-Gau · 2021-08-31T16:20:38Z

Tempted to keep those strain and strain_rrid in the modality agnostic part, because at least 2 BEP will be using them and if we only start allowing in this part of the spec things that can REALLY ONLY apply to any modality then, we might have to move handedness out as I am not sure how it applies to rodents, invertebrates, cell cultures...

SylvainTakerkart · 2021-09-01T07:50:55Z

Tempted to keep those strain and strain_rrid in the modality agnostic part, because at least 2 BEP will be using them and if we only start allowing in this part of the spec things that can REALLY ONLY apply to any modality then, we might have to move handedness out as I am not sure how it applies to rodents, invertebrates, cell cultures...

agreed! if we only put them in the modality-specific part, we will have to repeat it for several future modalities and it's going to get more complex to handle the specs...

mariehbourget · 2021-09-01T23:32:46Z

Thank you everyone for your feedback!
All the comments have been addressed and the PR is ready for re-review.

mariehbourget · 2021-10-08T17:47:25Z

Hi @bids-standard/maintainers,
All the comments in the PR have been addressed. I think it would be ready to be merged, is there something else we need to do?

I also wanted to mention that it may have an impact on another opened PR (#827) as it deals with TSV columns.
Thanks!

effigies · 2021-10-08T19:11:03Z

I am still +1 for merge. Any seconds?

tsalo

It looks good to me. I can handle adding the new columns to the TSV columns PR.

mariehbourget · 2021-10-13T13:32:52Z

Great! thank you @effigies and @tsalo, should I do the merge or are the PRs usually merged by maintainers?

effigies · 2021-10-13T14:37:27Z

Since the last meaningful commit was in August, I think we can merge. @sappelhoff are there any blocks I'm forgetting?

sappelhoff · 2021-10-13T18:17:51Z

Nope, I think this can go in 👍

* Add template. * Add first column. * Add columns. * More terms. * Fill name field in all files. * More work. Note that there are three very different uses of "name" columns, and two of them are equally common, so I chose not to specify any of them as the "canonical" definition. * Add remaining definitions. * Add macro to render column tables. * Fix YAML file. * Consolidate suffixes file. * Remove old individual files. * Move columns file. * Fix things up a bit. * Add columns I missed for modality-agnostic TSV files. * Support n/a for duration. * Apply suggestions from code review Co-authored-by: Chris Markiewicz <[email protected]> Co-authored-by: Stefan Appelhoff <[email protected]> * Code formatting in stim_file definition. * Allow numbers and strings for value. * Update src/schema/objects/columns.yaml Co-authored-by: Stefan Appelhoff <[email protected]> * Allow n/a for "z" column. Addresses https://github.com/bids-standard/bids-specification/pull/827/files#r723280787. * Describe meanings of x, y, and z columns. Addresses https://github.com/bids-standard/bids-specification/pull/827/files#r723283314. * Allow n/a for status column. Addresses https://github.com/bids-standard/bids-specification/pull/827/files#r723269382. * Add participant_id to participants.tsv table and append info for other IDs. * Split type definitions into channels and electrodes versions. * Update definitions for group based on file type. * Split reference column definition. * Clean up name_channels definition. * Draft new columns from #816 * Add new columns to table. * Remove list items. * Update src/04-modality-specific-files/04-intracranial-electroencephalography.md Co-authored-by: Stefan Appelhoff <[email protected]> * Apply suggestions from code review Co-authored-by: Chris Markiewicz <[email protected]> * Use two underscores to delineate multiply-defined columns. * Remove text that is now in table. * Update src/schema/objects/columns.yaml Co-authored-by: Chris Markiewicz <[email protected]> * Add sections to README on columns file and on reused terms. * Add EDF info to acq_time definition. * Remove hardcoded tables. * Remove unused links. Co-authored-by: Chris Markiewicz <[email protected]> Co-authored-by: Stefan Appelhoff <[email protected]>

Add recommended and optional columns to participants.tsv in

13b4c83

src/03-modality-agnostic-files.md

mariehbourget mentioned this pull request Jun 4, 2021

[ENH] BEP031 - New entity: sample and samples.tsv file #812

Merged

satra reviewed Jun 4, 2021

View reviewed changes

effigies reviewed Jun 6, 2021

View reviewed changes

Merge branch 'bids-standard:master' into bep031_participants_tsv

5bb1bf8

sappelhoff reviewed Jul 16, 2021

View reviewed changes

rob-luke reviewed Jul 16, 2021

View reviewed changes

mariehbourget and others added 4 commits July 16, 2021 10:07

Modify species as recommended and add examples

8b41137

Remove diagnosis column (out of scope for animal properties)

7146144

Add example to strain ans strain_rrid

b451671

Merge branch 'bids-standard:master' into bep031_participants_tsv

d08542f

mariehbourget added 2 commits August 5, 2021 10:51

Merge branch 'bids-standard:master' into bep031_participants_tsv

10316d5

Merge branch 'bids-standard:master' into bep031_participants_tsv

11a0ea1

Fix typo

4822ed0

Merge branch 'bids-standard:master' into bep031_participants_tsv

cb5a524

Merge branch 'bids-standard:master' into bep031_participants_tsv

393d3ce

mariehbourget mentioned this pull request Sep 16, 2021

[ENH] BEP031: Microscopy #881

Merged

8 tasks

effigies approved these changes Sep 28, 2021

View reviewed changes

Merge branch 'bids-standard:master' into bep031_participants_tsv

2a3bc86

tsalo approved these changes Oct 8, 2021

View reviewed changes

tsalo mentioned this pull request Oct 9, 2021

[SCHEMA] Add TSV column files #827

Merged

2 tasks

Merge branch 'bids-standard:master' into bep031_participants_tsv

7190c6e

effigies merged commit aae6274 into bids-standard:master Oct 13, 2021

tsalo added a commit to tsalo/bids-specification that referenced this pull request Oct 13, 2021

Draft new columns from bids-standard#816

552b4ae

mariehbourget added a commit that referenced this pull request Oct 15, 2021

Update "Recommended participant data" after merge of PR #816

a63b5b7

Remi-Gau mentioned this pull request Feb 8, 2022

Add sample entity and samples.tsv file #779

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ENH] BEP031 - New columns to participants.tsv file #816

[ENH] BEP031 - New columns to participants.tsv file #816

mariehbourget commented Jun 4, 2021

satra Jun 4, 2021

effigies Jun 6, 2021

mariehbourget Jun 7, 2021

satra Jun 8, 2021

mariehbourget Jul 16, 2021

satra Jun 4, 2021

mariehbourget Jun 7, 2021

mariehbourget Jul 16, 2021

effigies Jun 6, 2021

mariehbourget Jun 7, 2021

satra Jun 8, 2021

mariehbourget Jul 16, 2021

sappelhoff left a comment

rob-luke Jul 16, 2021

mariehbourget Jul 16, 2021

mariehbourget commented Jul 16, 2021

mariehbourget commented Aug 31, 2021

sappelhoff commented Aug 31, 2021

Remi-Gau commented Aug 31, 2021

SylvainTakerkart commented Sep 1, 2021

mariehbourget commented Sep 1, 2021

mariehbourget commented Oct 8, 2021 •

edited

Loading

effigies commented Oct 8, 2021

tsalo left a comment

mariehbourget commented Oct 13, 2021

effigies commented Oct 13, 2021

sappelhoff commented Oct 13, 2021

	When different from `homo sapiens`, `participants.tsv` SHOULD include a `species`
	When different from `homo sapiens`, `participants.tsv` MUST include a `species`

[ENH] BEP031 - New columns to participants.tsv file #816

[ENH] BEP031 - New columns to participants.tsv file #816

Conversation

mariehbourget commented Jun 4, 2021

Context

Contribution

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sappelhoff left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mariehbourget commented Jul 16, 2021

mariehbourget commented Aug 31, 2021

sappelhoff commented Aug 31, 2021

Remi-Gau commented Aug 31, 2021

SylvainTakerkart commented Sep 1, 2021

mariehbourget commented Sep 1, 2021

mariehbourget commented Oct 8, 2021 • edited Loading

effigies commented Oct 8, 2021

tsalo left a comment

Choose a reason for hiding this comment

mariehbourget commented Oct 13, 2021

effigies commented Oct 13, 2021

sappelhoff commented Oct 13, 2021

mariehbourget commented Oct 8, 2021 •

edited

Loading