[Bug]: kabooms on an elderly .nwb file

[yarikoptic]

What happened?

initially ran into it with dandi validate,

updated env

❯ pip install --upgrade nwbinspector hdmf pynwb
Requirement already satisfied: nwbinspector in /home/yoh/proj/dandi/dandi-cli-master/venvs/dev3/lib/python3.10/site-packages (0.4.17)
Collecting nwbinspector
  Downloading nwbinspector-0.4.23-py3-none-any.whl (41 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 41.5/41.5 KB 503.9 kB/s eta 0:00:00
Requirement already satisfied: hdmf in /home/yoh/proj/dandi/dandi-cli-master/venvs/dev3/lib/python3.10/site-packages (3.4.7)
Requirement already satisfied: pynwb in /home/yoh/proj/dandi/dandi-cli-master/venvs/dev3/lib/python3.10/site-packages (2.2.0)
Requirement already satisfied: jsonschema in /home/yoh/proj/dandi/dandi-cli-master/venvs/dev3/lib/python3.10/site-packages (from nwbinspector) (4.7.2)
Collecting isodate
  Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Requirement already satisfied: click in /home/yoh/proj/dandi/dandi-cli-master/venvs/dev3/lib/python3.10/site-packages (from nwbinspector) (8.1.3)
Requirement already satisfied: tqdm in /home/yoh/proj/dandi/dandi-cli-master/venvs/dev3/lib/python3.10/site-packages (from nwbinspector) (4.64.0)
Requirement already satisfied: numpy<1.23.0,>=1.22.0 in /home/yoh/proj/dandi/dandi-cli-master/venvs/dev3/lib/python3.10/site-packages (from nwbinspector) (1.22.4)
Requirement already satisfied: natsort in /home/yoh/proj/dandi/dandi-cli-master/venvs/dev3/lib/python3.10/site-packages (from nwbinspector) (8.1.0)
Requirement already satisfied: PyYAML in /home/yoh/proj/dandi/dandi-cli-master/venvs/dev3/lib/python3.10/site-packages (from nwbinspector) (6.0)
Requirement already satisfied: packaging in /home/yoh/proj/dandi/dandi-cli-master/venvs/dev3/lib/python3.10/site-packages (from nwbinspector) (21.3)
Requirement already satisfied: h5py<4,>=2.10 in /home/yoh/proj/dandi/dandi-cli-master/venvs/dev3/lib/python3.10/site-packages (from hdmf) (3.7.0)
Requirement already satisfied: pandas<2,>=1.0.5 in /home/yoh/proj/dandi/dandi-cli-master/venvs/dev3/lib/python3.10/site-packages (from hdmf) (1.4.3)
Requirement already satisfied: scipy<2,>=1.1 in /home/yoh/proj/dandi/dandi-cli-master/venvs/dev3/lib/python3.10/site-packages (from hdmf) (1.8.1)
Requirement already satisfied: setuptools in /home/yoh/proj/dandi/dandi-cli-master/venvs/dev3/lib/python3.10/site-packages (from hdmf) (59.6.0)
Requirement already satisfied: ruamel.yaml<1,>=0.16 in /home/yoh/proj/dandi/dandi-cli-master/venvs/dev3/lib/python3.10/site-packages (from hdmf) (0.17.21)
Requirement already satisfied: python-dateutil<3,>=2.7.3 in /home/yoh/proj/dandi/dandi-cli-master/venvs/dev3/lib/python3.10/site-packages (from pynwb) (2.8.2)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /home/yoh/proj/dandi/dandi-cli-master/venvs/dev3/lib/python3.10/site-packages (from jsonschema->nwbinspector) (0.18.1)
Requirement already satisfied: attrs>=17.4.0 in /home/yoh/proj/dandi/dandi-cli-master/venvs/dev3/lib/python3.10/site-packages (from jsonschema->nwbinspector) (21.4.0)
Requirement already satisfied: pytz>=2020.1 in /home/yoh/proj/dandi/dandi-cli-master/venvs/dev3/lib/python3.10/site-packages (from pandas<2,>=1.0.5->hdmf) (2022.1)
Requirement already satisfied: six>=1.5 in /home/yoh/proj/dandi/dandi-cli-master/venvs/dev3/lib/python3.10/site-packages (from python-dateutil<3,>=2.7.3->pynwb) (1.16.0)
Requirement already satisfied: ruamel.yaml.clib>=0.2.6 in /home/yoh/proj/dandi/dandi-cli-master/venvs/dev3/lib/python3.10/site-packages (from ruamel.yaml<1,>=0.16->hdmf) (0.2.6)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /home/yoh/proj/dandi/dandi-cli-master/venvs/dev3/lib/python3.10/site-packages (from packaging->nwbinspector) (3.0.9)
Installing collected packages: isodate, nwbinspector
  Attempting uninstall: nwbinspector
    Found existing installation: nwbinspector 0.4.17
    Uninstalling nwbinspector-0.4.17:
      Successfully uninstalled nwbinspector-0.4.17
Successfully installed isodate-0.6.1 nwbinspector-0.4.23

and plain nwbinspector call also kabooms

❯ nwbinspector Example.nwb
/home/yoh/proj/dandi/dandi-cli-master/venvs/dev3/lib/python3.10/site-packages/hdmf/backends/hdf5/h5tools.py:196: UserWarning: No cached namespaces found in Example.nwb
  warnings.warn(msg)
Traceback (most recent call last):
  File "/home/yoh/proj/dandi/dandi-cli-master/venvs/dev3/bin/nwbinspector", line 8, in <module>
    sys.exit(inspect_all_cli())
  File "/home/yoh/proj/dandi/dandi-cli-master/venvs/dev3/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/home/yoh/proj/dandi/dandi-cli-master/venvs/dev3/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/home/yoh/proj/dandi/dandi-cli-master/venvs/dev3/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/yoh/proj/dandi/dandi-cli-master/venvs/dev3/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/home/yoh/proj/dandi/dandi-cli-master/venvs/dev3/lib/python3.10/site-packages/nwbinspector/nwbinspector.py", line 269, in inspect_all_cli
    messages = list(
  File "/home/yoh/proj/dandi/dandi-cli-master/venvs/dev3/lib/python3.10/site-packages/nwbinspector/nwbinspector.py", line 401, in inspect_all
    nwbfile = robust_s3_read(io.read)
  File "/home/yoh/proj/dandi/dandi-cli-master/venvs/dev3/lib/python3.10/site-packages/nwbinspector/utils.py", line 171, in robust_s3_read
    raise exc
  File "/home/yoh/proj/dandi/dandi-cli-master/venvs/dev3/lib/python3.10/site-packages/nwbinspector/utils.py", line 166, in robust_s3_read
    return command(*command_args, **command_kwargs)
  File "/home/yoh/proj/dandi/dandi-cli-master/venvs/dev3/lib/python3.10/site-packages/hdmf/backends/hdf5/h5tools.py", line 451, in read
    return super().read(**kwargs)
  File "/home/yoh/proj/dandi/dandi-cli-master/venvs/dev3/lib/python3.10/site-packages/hdmf/utils.py", line 645, in func_call
    return func(args[0], **pargs)
  File "/home/yoh/proj/dandi/dandi-cli-master/venvs/dev3/lib/python3.10/site-packages/hdmf/backends/io.py", line 42, in read
    container = self.__manager.construct(f_builder)
  File "/home/yoh/proj/dandi/dandi-cli-master/venvs/dev3/lib/python3.10/site-packages/hdmf/utils.py", line 645, in func_call
    return func(args[0], **pargs)
  File "/home/yoh/proj/dandi/dandi-cli-master/venvs/dev3/lib/python3.10/site-packages/hdmf/build/manager.py", line 280, in construct
    result = self.__type_map.construct(builder, self, None)
  File "/home/yoh/proj/dandi/dandi-cli-master/venvs/dev3/lib/python3.10/site-packages/hdmf/utils.py", line 645, in func_call
    return func(args[0], **pargs)
  File "/home/yoh/proj/dandi/dandi-cli-master/venvs/dev3/lib/python3.10/site-packages/hdmf/build/manager.py", line 784, in construct
    obj_mapper = self.get_map(builder)
  File "/home/yoh/proj/dandi/dandi-cli-master/venvs/dev3/lib/python3.10/site-packages/hdmf/utils.py", line 645, in func_call
    return func(args[0], **pargs)
  File "/home/yoh/proj/dandi/dandi-cli-master/venvs/dev3/lib/python3.10/site-packages/hdmf/build/manager.py", line 703, in get_map
    container_cls = self.get_cls(obj)
  File "/home/yoh/proj/dandi/dandi-cli-master/venvs/dev3/lib/python3.10/site-packages/hdmf/utils.py", line 645, in func_call
    return func(args[0], **pargs)
  File "/home/yoh/proj/dandi/dandi-cli-master/venvs/dev3/lib/python3.10/site-packages/hdmf/build/manager.py", line 628, in get_cls
    raise ValueError("No data_type found for builder %s" % builder.path)
ValueError: No data_type found for builder root

file from http://github.com/NeurodataWithoutBorders/exp2nwb https://github.com/NeurodataWithoutBorders/exp2nwb/blob/master/example/Example.nwb

• I think it is worth channeling to hdmf and/or pynwb -- shouldn't kaboom I guess, right?
• is it expected from nwninspector to just bubble up exceptions from pynwb/hdmf? (I did mark the last checkbox below though ;))

Steps to Reproduce

see above

Traceback

note - -you give no choice for python 3.10

Operating System

Linux

Python Executable

Python

Python Version

3.9

Usage

Command Line Interface

Were you streaming with ROS3?

No

Package Versions

No response

Code of Conduct

• I agree to follow this project's Code of Conduct
• Have you ensured this bug was not already reported?
• To the best of your ability, have you ensured this is a bug within the code that checks the NWBFile, rather than a bug in the NWBFile reader (e.g., PyNWB or MatNWB)?

[oruebel]

Looking at the file, the nwb version is 1.0.5g_beta.

   DATASET "nwb_version" {
      DATATYPE  H5T_STRING {
         STRSIZE 11;
         STRPAD H5T_STR_NULLPAD;
         CSET H5T_CSET_ASCII;
         CTYPE H5T_C_S1;
      }
      DATASPACE  SCALAR
      DATA {
      (0): "1.0.5g_beta"
      }
   }

PyNWB and the Inspector do not support NWB 1 files. Also looking at the notebook with the code that generated the file, it looks like there are a number of validation errors even back when the file was generated https://github.com/NeurodataWithoutBorders/exp2nwb/blob/master/example/Example.ipynb .

[yarikoptic]

PyNWB and the Inspector do not support NWB 1 files.

Would be nice if pynwb then raised explicit exception, eg UnsupportedNWBVersionError or alike when encountering such a file

[oruebel]

Would be nice if pynwb then raised explicit exception,

Agreed. I create the following issue on PyNWB for this NeurodataWithoutBorders/pynwb#1609

[CodyCBakerPhD]

I think it is worth channeling to hdmf and/or pynwb -- shouldn't kaboom I guess, right?
is it expected from nwninspector to just bubble up exceptions from pynwb/hdmf? (I did mark the last checkbox below though ;))

It ought to have caught it and reported as an ERROR level event, yes. While PyNWB makes the error more informative (and by extension, the traceback message for the returned InspectorMessage), I'll also extend the range on the error capturing here.

[oruebel]

The following PR NeurodataWithoutBorders/pynwb#1612 now adds the function NWBHDF5IO.nwb_version to allow checking of file versions before reading the NWB file and adds a version check to NWBHDF5IO.read to raise a TypeError if an unsupported NWB version is found. The test on read can optionally be skipped by setting io.read(skip_version_check=False) but is enabled by default.