PYNWB_VALIDATION: missing data type Device

[laurelrr]

Quick question regarding the ast_ophys toolbox:
I ran the scripts to convert one subject's data and the output of the NWBInspector listed this:

0 PYNWB_VALIDATION

0.0 /snlkt/ast/ASt_NWB/NWB_Masters/calcium_ophys/nwbfiles/C6-J588-Disc5.nwb: ImagingPlane - 'None' object at location 'general/optophysiology/ImagingPlane'
Message: missing data type Device (device)

Is this important to fix? I haven't seen the pynwb validation flags before.

[CodyCBakerPhD]

PYNWB_VALIDATION issues are pretty important to fix, they are a level below 'ERRORS' and a level above 'CRITICAL'

There have been a few changes recently to how ImagingPlane and Device objects are linked in the NeuroConv metadata, so it does senstiively depend on the version of NeuroConv being used in your current environment

(a) what is pip show neuroconv in the environment used to create this file?

(b) I notice https://github.com/catalystneuro/tye-lab-to-nwb/blob/main/src/tye_lab_to_nwb/ast_ophys/requirements.txt#L2 doesn't pin an exact version, but maybe @weiglszonja remembers what to pin to?

(c) or we could try to quickly update the interface to use the new key-based linkage in the metadata

[laurelrr]

(a) pip show neuroconv:
Name: neuroconv
Version: 0.4.5
Summary: Convert data from proprietary formats to NWB format.
Home-page: https://github.com/catalystneuro/neuroconv
Author: Cody Baker, Szonja Weigl, Heberto Mayorquin, Luiz Tauffer, and Ben Dichter.
Author-email: [email protected]
License: BSD-3-Clause
Location: /home/lkeyes/anaconda3/envs/tye_lab_to_nwb_env/lib/python3.11/site-packages
Requires: dandi, fparse, h5py, hdmf, jsonschema, numpy, pandas, psutil, pynwb, PyYAML, scipy, tqdm
Required-by: tye-lab-to-nwb

[weiglszonja]

I think it is because ImagingPlane has a link for Device which is an extension of Device from ndx-miniscope, we opened an issue for this, which I see has not been resolved. @CodyCBakerPhD and I will try to push this, and I'll let you know once we have a solution for this.

[CodyCBakerPhD]

Yeah I remember this now. Core team has been aware of it for some time but they still don't have a fix/recommendation in place

I would say just ignore this for the time being @laurelrr, if you reach the DANDI upload step of the process and encounter the same problem at that level then there's a couple of back doors we can talk about trying to use at that point but for now, it's not really (and shouldn't really be) a problem

[CodyCBakerPhD]

I'm guessing this didn't cause a problem during upload @laurelrr?

[laurelrr]

Initially, I saw this error when running the NWB inspector:

**************************************************
NWBInspector Report Summary

Timestamp: 2024-05-21 14:29:41.373437-07:00
Platform: Linux-4.19.0-26-amd64-x86_64-with-glibc2.28
NWBInspector version: 0.4.35

Found 299 issues over 34 files:
       6 - ERROR
      33 - BEST_PRACTICE_VIOLATION
     260 - BEST_PRACTICE_SUGGESTION
**************************************************


0  ERROR
========

0.0  H27_Disc4.nwb and 2 other files: check_session_start_time_old_date - 'None' object with name 'None'
       Message: Traceback (most recent call last):
  File "/home/lkeyes/anaconda3/envs/pynwb/lib/python3.10/site-packages/nwbinspector/nwbinspector.py", line 726, in run_checks
    output = robust_s3_read(command=check_function, command_args=[nwbfile_object])
  File "/home/lkeyes/anaconda3/envs/pynwb/lib/python3.10/site-packages/nwbinspector/utils.py", line 174, in robust_s3_read
    raise exc
  File "/home/lkeyes/anaconda3/envs/pynwb/lib/python3.10/site-packages/nwbinspector/utils.py", line 169, in robust_s3_read
    return command(*command_args, **command_kwargs)
  File "/home/lkeyes/anaconda3/envs/pynwb/lib/python3.10/site-packages/nwbinspector/register_checks.py", line 128, in auto_parse_some_output
    output = check_function(*args, **kwargs)
  File "/home/lkeyes/anaconda3/envs/pynwb/lib/python3.10/site-packages/nwbinspector/checks/nwbfile_metadata.py", line 31, in check_session_start_time_old_date
    if session_start_time.tzinfo is not None:
AttributeError: 'datetime.date' object has no attribute 'tzinfo'


0.1  H27_Disc4.nwb and 2 other files: check_session_start_time_future_date - 'None' object with name 'None'
       Message: Traceback (most recent call last):
  File "/home/lkeyes/anaconda3/envs/pynwb/lib/python3.10/site-packages/nwbinspector/nwbinspector.py", line 726, in run_checks
    output = robust_s3_read(command=check_function, command_args=[nwbfile_object])
  File "/home/lkeyes/anaconda3/envs/pynwb/lib/python3.10/site-packages/nwbinspector/utils.py", line 174, in robust_s3_read
    raise exc
  File "/home/lkeyes/anaconda3/envs/pynwb/lib/python3.10/site-packages/nwbinspector/utils.py", line 169, in robust_s3_read
    return command(*command_args, **command_kwargs)
  File "/home/lkeyes/anaconda3/envs/pynwb/lib/python3.10/site-packages/nwbinspector/register_checks.py", line 128, in auto_parse_some_output
    output = check_function(*args, **kwargs)
  File "/home/lkeyes/anaconda3/envs/pynwb/lib/python3.10/site-packages/nwbinspector/checks/nwbfile_metadata.py", line 48, in check_session_start_time_future_date
    if session_start_time.tzinfo is not None:
AttributeError: 'datetime.date' object has no attribute 'tzinfo'

Next, I re-converted the data for the subjects with the error using the updated pynwb version.
When I re-ran the inspector on the new files, these errors were gone:

**************************************************
NWBInspector Report Summary

Timestamp: 2024-05-21 15:30:18.887502-07:00
Platform: Linux-4.19.0-26-amd64-x86_64-with-glibc2.28
NWBInspector version: 0.4.35

Found 9 issues over 1 files:
       1 - BEST_PRACTICE_VIOLATION
       8 - BEST_PRACTICE_SUGGESTION
**************************************************


0  BEST_PRACTICE_VIOLATION
==========================

0.0  H27_Disc4.nwb: check_image_series_external_file_relative - 'ImageSeries' object at location '/acquisition/Video: H027Disc4'
       Message: The external file '/snlkt/data/hao/Neurotensin/ephys/recordings/PVT-BLA_NT_KO_CeMproj batch1/videos/H027Disc4.mkv' is not a relative path. Please adjust the absolute path to be relative to the location of the NWBFile.

1  BEST_PRACTICE_SUGGESTION
===========================

1.1  H27_Disc4.nwb: check_description - 'Subject' object at location '/general/subject'
       Message: Description is missing.

1.2  H27_Disc4.nwb: check_description - 'Device' object at location '/general/devices/DeviceEcephys'
       Message: Description ('no description') is a placeholder.

1.3  H27_Disc4.nwb: check_description - 'ElectrodeGroup' object at location '/general/extracellular_ephys/ElectrodeGroup'
       Message: Description ('no description') is a placeholder.

1.4  H27_Disc4.nwb: check_description - 'VectorData' object with name 'gain_to_uV'
       Message: Description ('no description') is a placeholder.

1.5  H27_Disc4.nwb: check_description - 'VectorData' object with name 'offset_to_uV'
       Message: Description ('no description') is a placeholder.

1.6  H27_Disc4.nwb: check_description - 'VectorData' object with name 'channel_name'
       Message: Description ('no description') is a placeholder.

1.7  H27_Disc4.nwb: check_description - 'PoseEstimation' object at location '/processing/behavior/PoseEstimation'
       Message: Description is missing.

1.8  H27_Disc4.nwb: check_keywords - 'NWBFile' object at location '/'
       Message: Metadata /general/keywords is missing.

I did not see any errors pop up during the dandi validation or dandi upload steps.

[laurelrr]

Earlier today, I added the fiber photometry files to the dandiset and that is when I saw the 5 asset validation errors.

[CodyCBakerPhD]

Great - that one remaining violation is actually taken care of by dandi organize (if you reran the inspector on the results of the organization; which dandi validate effectively does)

Glad to see the original device issue is resolved