Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: untyped dataset of references cannot be set as expandable #1181

Closed
stephprince opened this issue Aug 29, 2024 · 0 comments · Fixed by #1188
Closed

[Bug]: untyped dataset of references cannot be set as expandable #1181

stephprince opened this issue Aug 29, 2024 · 0 comments · Fixed by #1188
Assignees
Labels
category: bug errors in the code or code behavior priority: medium non-critical problem and/or affecting only a small set of users topic: PyNWB Issues related to the use HDMF in PyNWB

Comments

@stephprince
Copy link
Contributor

What happened?

With appending to an electrodes table, the group column cannot be set as expandable.

I have a working solution to this and just need to add the tests.

Steps to Reproduce

from datetime import datetime
from uuid import uuid4
from dateutil import tz

from pynwb import NWBHDF5IO, NWBFile, H5DataIO

session_start_time = datetime(2018, 4, 25, 2, 30, 3, tzinfo=tz.gettz("US/Pacific"))

# setup NWBFile
nwbfile = NWBFile(
    session_description="Mouse exploring an open field",  # required
    identifier=str(uuid4()),  # required
    session_start_time=session_start_time,  # required
)

# add column
device = nwbfile.create_device(name="array", description="an array", manufacturer="company")

# create an electrode group
electrode_group = nwbfile.create_electrode_group(
    name="shank0",
    description="electrode group for shank 0",
    device=device,
    location="brain area",
)

# add electrode to the electrode table
nwbfile.add_electrode(group=electrode_group, location="brain area",)
nwbfile.electrodes.id.set_data_io(H5DataIO, {'maxshape': (None,), 'chunks': True})
nwbfile.electrodes.group.set_data_io(H5DataIO, {'maxshape': (None,), 'chunks': True})
nwbfile.electrodes.group_name.set_data_io(H5DataIO, {'maxshape': (None,), 'chunks': True})
nwbfile.electrodes.location.set_data_io(H5DataIO, {'maxshape': (None,), 'chunks': True})

with NWBHDF5IO("test_append_electrodes_table.nwb", "w") as io:
    io.write(nwbfile)

with NWBHDF5IO("test_append_electrodes_table.nwb", mode="a") as io:
    nwbfile = io.read()
    electrode_group = nwbfile.electrode_groups["shank0"]
    
    nchannels = 4
    for ielec in range(nchannels):
        nwbfile.add_electrode(
            group=electrode_group,
            location="brain area",
        )

    io.write(nwbfile)

with NWBHDF5IO("test_append_electrodes_table.nwb", "r") as io:
    nwbfile = io.read()
    print(nwbfile.electrodes.to_dataframe())

Traceback

Traceback (most recent call last):
  File "/Users/smprince/anaconda3/envs/pynwb/lib/python3.11/runpy.py", line 198, in _run_module_as_main
    return _run_code(code, main_globals, None,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/smprince/anaconda3/envs/pynwb/lib/python3.11/runpy.py", line 88, in _run_code
    exec(code, run_globals)
  File "/Users/smprince/.vscode/extensions/ms-python.debugpy-2024.10.0-darwin-arm64/bundled/libs/debugpy/adapter/../../debugpy/launcher/../../debugpy/__main__.py", line 39, in <module>
    cli.main()
  File "/Users/smprince/.vscode/extensions/ms-python.debugpy-2024.10.0-darwin-arm64/bundled/libs/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 430, in main
    run()
  File "/Users/smprince/.vscode/extensions/ms-python.debugpy-2024.10.0-darwin-arm64/bundled/libs/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 284, in run_file
    runpy.run_path(target, run_name="__main__")
  File "/Users/smprince/.vscode/extensions/ms-python.debugpy-2024.10.0-darwin-arm64/bundled/libs/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 321, in run_path
    return _run_module_code(code, init_globals, run_name,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/smprince/.vscode/extensions/ms-python.debugpy-2024.10.0-darwin-arm64/bundled/libs/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 135, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/Users/smprince/.vscode/extensions/ms-python.debugpy-2024.10.0-darwin-arm64/bundled/libs/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 124, in _run_code
    exec(code, run_globals)
  File "/Users/smprince/Documents/code/pynwb/issue_scripts/electrode_table_testing.py", line 43, in <module>
    nwbfile.add_electrode(
  File "/Users/smprince/Documents/code/hdmf/src/hdmf/utils.py", line 668, in func_call
    return func(args[0], **pargs)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/smprince/Documents/code/pynwb/src/pynwb/file.py", line 725, in add_electrode
    self.electrodes.add_row(**d)
  File "/Users/smprince/Documents/code/hdmf/src/hdmf/utils.py", line 668, in func_call
    return func(args[0], **pargs)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/smprince/Documents/code/hdmf/src/hdmf/common/table.py", line 715, in add_row
    c.add_row(data[colname])
  File "/Users/smprince/Documents/code/hdmf/src/hdmf/utils.py", line 668, in func_call
    return func(args[0], **pargs)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/smprince/Documents/code/hdmf/src/hdmf/common/table.py", line 52, in add_row
    self.append(val)
  File "/Users/smprince/Documents/code/hdmf/src/hdmf/container.py", line 948, in append
    self.__data = append_data(self.__data, arg)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/smprince/Documents/code/hdmf/src/hdmf/data_utils.py", line 23, in append_data
    data.append(arg)
  File "/Users/smprince/Documents/code/hdmf/src/hdmf/backends/hdf5/h5_utils.py", line 123, in append
    append_data(self.dataset, ref)
  File "/Users/smprince/Documents/code/hdmf/src/hdmf/data_utils.py", line 36, in append_data
    data.resize(shape)
  File "/Users/smprince/anaconda3/envs/pynwb/lib/python3.11/site-packages/h5py/_hl/dataset.py", line 666, in resize
    raise TypeError("Only chunked datasets can be resized")
TypeError: Only chunked datasets can be resized

Operating System

macOS

Python Executable

Conda

Python Version

3.11

Package Versions

No response

@stephprince stephprince self-assigned this Aug 29, 2024
@stephprince stephprince added category: bug errors in the code or code behavior priority: medium non-critical problem and/or affecting only a small set of users topic: PyNWB Issues related to the use HDMF in PyNWB labels Aug 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
category: bug errors in the code or code behavior priority: medium non-critical problem and/or affecting only a small set of users topic: PyNWB Issues related to the use HDMF in PyNWB
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant