-
Notifications
You must be signed in to change notification settings - Fork 3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Synchronization (or tracking of offset) of involved clocks #13
Comments
so the idea would be to create a "calibration stimuli script" which would
based on that data AND timestamps in the DICOMs we would be able to tell the offset and from multiple (across weeks) acquisitions to tell the drift of the clock in the MR reconstruction box. Then this information on clock differences would be used for partitioning videos (#14). |
more on this idea (which I like now even more): by presenting it from reproin'er (the same box which also has video capturing dongle attached) we can assess delay introduced by the capturing dongle! (assuming that stimuli script does all the timing correctly) |
proceed after #35 is done. |
in preparation to tomorrow QA/sync acquisition -- @andycon please prepare a script which would keep presenting QR codes (e.g. every 10th frame or so?) with that datetime up to milliseconds embedded so we could later assess delay/jitter between video stimuli delivery and capture (in addition to magnet triggers captured by #35) |
summary of times for the sample `006-func-bold_task-rest_run-1` acquisition and only 1st dynamic shown here: not ultimately usable since we are lacking qrcode recording for some reason
❯ grep acqNum 20240528_run-1.log
{"event": "trigger", "acqNum": 0, "logfn": "20240528-2.log", "time": 1716908819.6869357, "time_formatted": "2024-05-28T11:06:59.686936-04:00", "keys": ["5"], "keys_time": 1716908832.3484943, "keys_time_str": "2024-05-28T11:07:12.348494-04:00", "time_flip": 1716908832.4185193, "time_flip_formatted": "2024-05-28T11:07:12.418519-04:00"}
❯ grep -A1 024-05-28T15:08 *
{"epoch_time": 1716908920.407058, "iso_time": "2024-05-28T15:08:40.407058+00:00", "time": 386.6154779999997, "alink_byte": 240, "alink_flags": 1}
{"epoch_time": 1716908940.610456, "iso_time": "2024-05-28T15:09:00.610456+00:00", "time": 406.82571899999994, "alink_byte": 496, "alink_flags": 2}
So redoing for the _run-2 for which we should have everything (yet TODO)
so we started to acquire first slice of the first volume at t_mri=11:15:11,502500.
ses-20240528/psychopy
❯ grep 'keys": \["5' 20240528_run-2.log | head -n 1
{"event": "trigger", "acqNum": 0, "logfn": "20240528_run-2.log", "time": 1716908929.1446826, "time_formatted": "2024-05-28T11:08:49.144683-04:00", "keys": ["5"], "keys_time": 1716908940.5812862, "keys_time_str": "2024-05-28T11:09:00.581286-04:00", "time_flip": 1716908940.6707587, "time_flip_formatted": "2024-05-28T11:09:00.670759-04:00"} so we received trigger pulse at t_experimenter=11:09:00.581286 (6 minutes diff!)
so we got that record on frame 639 (time from video start yet to be determined to establish relationship to t_reproiner etc) for 18 frames.
ses-20240528/birch
❯ grep -A1 -E '15:(0[789]|1[0123]):.*alink_byte.*496' 20240528-105648.jsonl
.... there came bunch for run-01, then break in time/lines and beginning of run-02: ...
{"epoch_time": 1716908940.610456, "iso_time": "2024-05-28T15:09:00.610456+00:00", "time": 406.82571899999994, "alink_byte": 496, "alink_flags": 2} so we have 11:09:00.610456 in t_birch
❯ grep -A1 -E '11:(09|1[0123])' 2024-05-28T11.csv | head -n 2
224,1,1716908940.6144216,2024-05-28T11:09:00.614422-04:00,6,0.000224,122.171526,-ACK,,1162414
219,0,1716908940.6344159,2024-05-28T11:09:00.634416-04:00,6,0.000219,122.191516,-ACK,,1162415 so 200ms pulse at 11:09:00.614422 in t_reproiner |
FWIW we have got a new data sample (again from my laptop :-/ ) which was pushed to typhon, full content of repoyoh@typhon:/data/repronim/reproflow-data-sync$ find * -maxdepth 2
code
code/collect_data.sh
ses-20240528
ses-20240528/DICOMS
ses-20240528/DICOMS/001-anat-scout_ses-{date}
ses-20240528/DICOMS/002-anat-scout_ses-{date}_MPR_sag
ses-20240528/DICOMS/003-anat-scout_ses-{date}_MPR_cor
ses-20240528/DICOMS/004-anat-scout_ses-{date}_MPR_tra
ses-20240528/DICOMS/005-func-bold_task-rest_run-1
ses-20240528/DICOMS/006-func-bold_task-rest_run-1
ses-20240528/DICOMS/007-func-bold_task-rest_run-2
ses-20240528/README.md
ses-20240528/psychopy
ses-20240528/psychopy/20240528-1.log
ses-20240528/psychopy/20240528_run-1.log
ses-20240528/psychopy/20240528_run-2.log
ses-20240528/psychopy/qr_code_flips.py
ses-20240528/reprostim-videos
ses-20240528/reprostim-videos/2024.05.28.11.08.50.176_2024.05.28.11.10.31.855.mkv
ses-20240528/reprostim-videos/2024.05.28.11.08.50.176_2024.05.28.11.10.31.855.mkv.log
ses-20240528/reprostim-videos/2024.05.28.11.11.35.277_2024.05.28.11.12.06.156.mkv
ses-20240528/reprostim-videos/2024.05.28.11.11.35.277_2024.05.28.11.12.06.156.mkv.log
ses-20240528/reprostim-videos/2024.05.28.11.18.01.667_2024.05.28.11.18.47.953.mkv
ses-20240528/reprostim-videos/2024.05.28.11.18.01.667_2024.05.28.11.18.47.953.mkv.log
ses-20240528/reprostim-videos/2024.05.28.11.01.49.915_2024.05.28.11.08.26.672.mkv
ses-20240528/reprostim-videos/2024.05.28.11.01.49.915_2024.05.28.11.08.26.672.mkv.log
ses-20240528/reprostim-videos/2024.05.28.11.01.49.915_2024.05.28.11.08.26.672.mkv-qr.jsonl
ses-20240528/reprostim-videos/2024.05.28.11.08.37.837_2024.05.28.11.08.44.617.mkv
ses-20240528/reprostim-videos/2024.05.28.11.08.37.837_2024.05.28.11.08.44.617.mkv-qr.jsonl
ses-20240528/reprostim-videos/2024.05.28.11.08.37.837_2024.05.28.11.08.44.617.mkv.log
ses-20240528/reprostim-videos/2024.05.28.11.08.50.176_2024.05.28.11.10.31.855.mkv-qr.jsonl
ses-20240528/reprostim-videos/2024.05.28.11.11.35.277_2024.05.28.11.12.06.156.mkv-qr.jsonl
ses-20240528/reprostim-videos/2024.05.28.11.18.01.667_2024.05.28.11.18.47.953.mkv-qr.jsonl
ses-20240528/reproevents
ses-20240528/reproevents/2024-05-28T11.csv
ses-20240528/reproevents/fetch.sh
ses-20240528/birch
ses-20240528/birch/20240528-105648.jsonl
ses-20240604
ses-20240604/DICOMS
ses-20240604/DICOMS/001-anat-scout_ses-{date}
ses-20240604/DICOMS/002-anat-scout_ses-{date}_MPR_sag
ses-20240604/DICOMS/003-anat-scout_ses-{date}_MPR_cor
ses-20240604/DICOMS/004-anat-scout_ses-{date}_MPR_tra
ses-20240604/DICOMS/005-func-bold_task-rest_run-1
ses-20240604/DICOMS/006-func-bold_task-rest_run-2
ses-20240604/DICOMS/007-func-bold_task-rest_run-3
ses-20240604/DICOMS/008-func-bold_task-rest_run-4
ses-20240604/README.md
ses-20240604/birch
ses-20240604/birch/out.jsonl
ses-20240604/psychopy
ses-20240604/psychopy/20240604_run-1.log
ses-20240604/psychopy/20240604_run-2.log
ses-20240604/psychopy/20240604_run-3.log
ses-20240604/psychopy/20240604_run-4.log
ses-20240604/psychopy/qr_code_flips.py
ses-20240604/reproevents
ses-20240604/reproevents/events.csv
ses-20240604/reprostim-videos
ses-20240604/reprostim-videos/2024.06.04.13.51.24.278_2024.06.04.13.51.31.057.mkv
ses-20240604/reprostim-videos/2024.06.04.13.51.24.278_2024.06.04.13.51.31.057.mkv.log
ses-20240604/reprostim-videos/2024.06.04.13.51.36.620_2024.06.04.13.58.20.763.mkv
ses-20240604/reprostim-videos/2024.06.04.13.51.36.620_2024.06.04.13.58.20.763.mkv.log but to facilitate collaboration etc I now pushed that dataset also to https://datasets.datalad.org/?dir=/repronim/reproflow-data-sync , so you can |
First round done:
/reprostim/Parsing$ ./parse_wQR.py --log-level INFO /data/repronim/reproflow-data-sync/ses-20240604/reprostim-videos/2024.06.04.13.51.36.620_2024.06.04.13.58.20.763.mkv > 2024-06-27.info.txt
|
Cool, thank you! Most likely we would also want some json output mode so we have it machine readable |
Lame description of setup to non MRI people: Each behavioral MRI experiment consists of multiple "anatomical scans" and "functional scans". "Anatomical scan" typically consists of 1 3D image of the brain (takes minutes to acquire). "Functional scan" is a series of 3D images (of lower resolution and much faster, typically 1-2 seconds per each). For each 3D "functional image" MRI sends a "trigger pulse" to inform that MRI starts acquiring new 3D volume. Experimenter's stimuli script typically awaits for that trigger pulse to come either just to start the entire "functional scan" or each volume/response acquisition.
edit: the description of the format (original, before we made it into json lines, while preserving all the information, just using regular decimal ( |
For calibration we will use our
With that we should be able to
In the dumps of data above, we use qr_code_flips.py script. Next step would be to write script (under We will assume that there is
|
Also executed latest parse_wQR.py @typhon
and placed JSON results to |
video stimuli grabber computer uses NTP to keep its clock nicely synchronized (related #12), but we do not know (yet) what is happening on MRI system which includes multiple computers (console, reconstruction server)
reproiner
receive trigger pulses directly from the scanner, so we could use those time stamps (in the clock of reproiner) to later better judge about beginning/end of acquisitionDetails of our current setup
and with more annotations:
with added connection from curdes to personal computer via USB (not yet committed here):
Regarding "what is where", here is a list of involved "computers"
reprostim
account:bids
accountupdate-lists
dailyreproiner
not yet used/deployed
On 2024/05/28 we got sample collection of data after syncing birch via ntp to reproiner and running the "show qr code with details" after each trigger pulse was received. Data is available there on typhon and here it is with added short descriptions
birch -- jsonlines records as obtained from modified (not open source yet) software on birch
vd
). First there was session where Terry tried all buttons on response box, so it looks likeso we have bit for 256 going into 1 and then back to 0.
DICOMS -- fMRI sequences on a phantom in DICOMs (not converted to nii.gz yet)
dcmdump
fromdcmtk
package to dump all metadataParsing/parse_wQR.py
extracts recorded as QR codes records which must correspond to the ones logged underpsychopy/
folderthere were 1 screwed up (I think) run and then 2 good ones. "good ones" should have
_run-1
and_run-2
where "available" (DICOM series description and psychopy logs have it for sure)The text was updated successfully, but these errors were encountered: