[BUG] erroneously deleted files SUSPICIOUSLY_LONG_EVENT_DESIGN

[jungheejung]

Which module is this from?

beh, func

What is the issue?

	[WARNING] SUSPICIOUSLY_LONG_EVENT_DESIGN The onset of the last event is after the total duration of the corresponding scan.
This design is suspiciously long.

		/sub-0004/ses-02/func/sub-0004_ses-02_task-faces_acq-mb8_run-01_bold.nii.gz
		/sub-0005/ses-04/func/sub-0005_ses-04_task-fractional_acq-mb8_run-01_bold.nii.gz

		12 more files with the same issue

What was your expected behavior?

/sub-0004/ses-02/func/sub-0004_ses-02_task-faces_acq-mb8_run-01_bold.nii.gz

How can we reproduce this?

(Write your answer here.)

Any additional context?

@yarikoptic I believe we checked the unredacted branch. Could we revisit this on meeting of Sep 20th?

[yarikoptic]

❯ jq -r '.issues.issues[] | select(.code == "SUSPICIOUSLY_LONG_EVENT_DESIGN") | .location' deno-bids-validator.json

/sub-0004/ses-02/func/sub-0004_ses-02_task-faces_acq-mb8_run-01_bold.nii.gz
/sub-0005/ses-04/func/sub-0005_ses-04_task-fractional_acq-mb8_run-01_bold.nii.gz
/sub-0011/ses-02/func/sub-0011_ses-02_task-narratives_acq-mb8_run-02_bold.nii.gz
/sub-0013/ses-04/func/sub-0013_ses-04_task-fractional_acq-mb8_run-01_bold.nii.gz
/sub-0017/ses-02/func/sub-0017_ses-02_task-alignvideo_acq-mb8_run-01_bold.nii.gz
/sub-0017/ses-04/func/sub-0017_ses-04_task-social_acq-mb8_run-04_bold.nii.gz
/sub-0025/ses-03/func/sub-0025_ses-03_task-shortvideo_acq-mb8_run-01_bold.nii.gz
/sub-0026/ses-03/func/sub-0026_ses-03_task-shortvideo_acq-mb8_run-01_bold.nii.gz
/sub-0035/ses-01/func/sub-0035_ses-01_task-alignvideo_acq-mb8_run-02_bold.nii.gz
/sub-0055/ses-02/func/sub-0055_ses-02_task-narratives_acq-mb8_run-04_bold.nii.gz
/sub-0061/ses-01/func/sub-0061_ses-01_task-alignvideo_acq-mb8_run-03_bold.nii.gz
/sub-0069/ses-02/func/sub-0069_ses-02_task-narratives_acq-mb8_run-03_bold.nii.gz
/sub-0084/ses-03/func/sub-0084_ses-03_task-alignvideo_acq-mb8_run-01_bold.nii.gz
/sub-0126/ses-01/func/sub-0126_ses-01_task-alignvideo_acq-mb8_run-01_bold.nii.gz

[jungheejung]

TRUE ERROR: RESOLVED SWAPPED DUPS

• /sub-0004/ses-02/func/sub-0004_ses-02_task-faces_acq-mb8_run-01_bold.nii.gz DUP was primary RESOLVED

EXPLORE

• /sub-0005/ses-04/func/sub-0005_ses-04_task-fractional_acq-mb8_run-01_bold.nii.gz DUP EXISTS; TR length is 1323 which checks out

TRUE ERROR: DO NOTHING but these runs are shorter than expected - Why is the error flagging Long event design?

• The listed runs are shorter than the intended TR length
• Confirmed that there are no dups via git log --stat remotes/rolando-exchange/unredacted-master -- "$FILE"
• ROOT CAUSE: event files are loner than the recorded BOLD. Primarily because some parameters are built in . we may have to retroactively remove rows

• /sub-0013/ses-04/func/sub-0013_ses-04_task-fractional_acq-mb8_run-01_bold.nii.gz → TRUE ERROR TR length 1234 < 1322 NO DUP

NOTES: - Fractionals Run01: in the last minute of the task the subject stopped the scan with the emergency ball and said her answers weren't recording. We discovered that the trackball right button was stuck and not recording responses at all. We had to swap out the trackball with the upstairs one.

• Further, the biopac network connection was not working properly, so we did not collect biopac data for fractionals. We did collect biopac for all remaining tasks.

• /sub-0017/ses-02/func/sub-0017_ses-02_task-alignvideo_acq-mb8_run-01_bold.nii.gz → TRUE ERROR341 < 1073; NO DUP

NOTES: "- SCANNER ERROR during Align Videos Run 01. We had to remove participant, save log files, reboot, put participant back in which took about 40 mins in all. We skipped the rest of Run 01.

• Align Video Run 01 and only ran runs 2-4 because P needed to leave.
  "
  EXECUTIVE DECISION: delete run-01?

• /sub-0017/ses-04/func/sub-0017_ses-04_task-social_acq-mb8_run-04_bold.nii.gz → TRUE ERROR 440 < 872; NO DUP

NOTES: - In general: LOTS of movement, multiple reminders to hold still
-PVC: run 4 (2nd cog) the scan ended while the task was still running

• /sub-0035/ses-01/func/sub-0035_ses-01_task-alignvideo_acq-mb8_run-02_bold.nii.gz → TRUE ERROR TR 489 < 1376; NO DUP

NOTES: - Align video: scan stopped early (during video with old man/2kids entering suprise party); missing imaging data from ~2.5 videos

• /sub-0055/ses-02/func/sub-0055_ses-02_task-narratives_acq-mb8_run-04_bold.nii.gz → TRUE ERROR 1126 < 1156 NO DUP

NOTES: none

• /sub-0061/ses-01/func/sub-0061_ses-01_task-alignvideo_acq-mb8_run-03_bold.nii.gz → TRUE ERROR 621 < 1016 NO DUP

NOTES: - Align vid: the scanner stopped close to the end of run 3 (last video [erotic cartoon] due to scanner hot water temperature error. Terry fixed it but we moved on to the 4th run.

• /sub-0069/ses-02/func/sub-0069_ses-02_task-narratives_acq-mb8_run-03_bold.nii.gz → TRUE ERROR 652 < 1298 NO DUP

Narratives: participant had to discontinue run 3 because of issue with glasses. Partial data collected.

• /sub-0084/ses-03/func/sub-0084_ses-03_task-alignvideo_acq-mb8_run-01_bold.nii.gz → TRUE ERROR TR: 520 < 1157 NO DUP

NOTES: "PVC: skipped pain runs (03+06) due to 8007 error. Details: the TSA had passed the safety test and successfully ran with matlab pretest before setting up the participant. When starting the first pain run, I noticed medoc wasn't making the usual trigger sound but the participant was still making ratings (she later confirmed verbally she did not feel pain stimulus). I pulled up medoc and saw that everything was greyed out and there was an error message about failing the pre-test even though the TSA unit itself was still green. I restarted matlab, stopped the run, and detached the thermode from the participant before redoing the safety test, which is when 8007 error first appeared.
Align videos: participant hit the emergency ball to report that the trackball stopped working properly. She verified that it had been working up until that point. We re-calibrated and went ahead to the next run. Issue was reported a little less than halfway through (circa the car flood video)."

FALSE ALARM

: Run length exactly matches the intended TR length. Not sure why the validator raised flags on this bold file.

• Solution: we can add the listed files in a bidsignore
• Future TODO, Not urgent:It may be worth identifying why the validator raises a false alarm.

• /sub-0011/ses-02/func/sub-0011_ses-02_task-narratives_acq-mb8_run-02_bold.nii.gz → FALSE ALARM: intended runlength is indeed 1098
• /sub-0025/ses-03/func/sub-0025_ses-03_task-shortvideo_acq-mb8_run-01_bold.nii.gz → FALSE ALARM: intended runlength is indeed 1616
• /sub-0026/ses-03/func/sub-0026_ses-03_task-shortvideo_acq-mb8_run-01_bold.nii.gz → FALSE ALARM: intended runlength is indeed 1616
• /sub-0126/ses-01/func/sub-0126_ses-01_task-alignvideo_acq-mb8_run-01_bold.nii.gz → FALSE intended runlength is indeed ALARM: 1073 NO DUP

[jungheejung]

@yarikoptic

• Update: I confirm that there are no cases where DUP was primary other than sub-0004
• Reason behind error: The error occurs because the events.tsv file contains onset times that exceed the recorded TR length (dcmmeta_shape[-1]). This typically happens when the BOLD acquisition ends earlier than expected, but the behavioral data remains intact. In such cases, the events.tsv may appear "suspiciously longer than expected" since the parameters were pre-determined and populated in advance for the experiment.
• Moving forward: When BOLD is shorter than events.tsv, should we retroactively remove the rows in events.tsv to match the length of BOLD file?

[jungheejung]

• cross-check scannotes. The shorter BOLD -- 8 runs -- are due to scanner or equipment failure. I have to make the decision of whether we shave off rows in events.tsv to match the BOLD data, or we remove them entirely.
• Another way is to fill rows with "n/a" and keep the rows of events.tsv instead of deleting them. Any thoughts? @yarikoptic
• [ ]

[yarikoptic]

if you know that there were no events for data points, I would just remove those events from _events files

[jungheejung]

@yarikoptic Thanks for the quick response!

There are some arbitrary cases - all ears to hear your thoughts.
sub-0013_ses-04_task-fractional_acq-mb8_run-01_events.tsv for example

• Expected scan length is 1322; * Total scan length is 1234
• According to the events.tsv, Participant stopped responding around scan length 667, basing off of when event duration is near 0
• Total event.tsv goes until 583.79 seconds, which amounts to 1269 scan images.

My intuition is remove events.tsv rows where participants do not respond, but this will lead in a discrepancy between BOLD and events.tsv either way.