Skip to content

Commit

Permalink
Stop IPFX from caching its NWB Schemas when writing/modifying NWB files
Browse files Browse the repository at this point in the history
When IPFX modifies existing NWB files produced by MIES/IPNWB
to add spike/metadata info, it adds additional pyNWB schemas
to `core` and `hdmf-common` that are newer version than the ones
the MIES/IPNWB uses. This has caused some grief when trying to
DANDI validate the updated NWB files, so we will just not cache
the updated schemas to the NWB file when IPFX interacts with the
NWB files.

*WARNING* This change will introduce fragility down the road if
IPFX writes NWB fields that require the newer schema version. Since
those newer schema versions will no longer be stored with the NWB
file, they may become 'unreadable'. This is not a problem currently
though since the older schema versions appear to support everything
written when adding spikes/metadata via IPFX.

*NOTE* pyNWB really doesn't support multiple nwb schemas for one
NWB file. The *right* long-term solution would be to ensure that
MIES/IPNWB and IPFX use the same version of nwb schemas like `core`
and `hdmf-common`.
  • Loading branch information
njmei committed Jun 29, 2022
1 parent 905ee74 commit f86ba1c
Show file tree
Hide file tree
Showing 2 changed files with 10 additions and 2 deletions.
6 changes: 5 additions & 1 deletion ipfx/attach_metadata/sink/nwb2_sink.py
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,11 @@ def _commit_nwb_changes(self):

set_container_sources(self.nwbfile, self._h5_file.filename)
self.nwbfile.set_modified(True)
self._nwb_io.write(self.nwbfile)
# Because the NWB schema versions of NWB data produced by MIES are older
# we do not want to cache the newer schema versions that IPFX is currently using
# WARNING: Doing this may introduce fragility down the road though if IPFX writes NWB fields
# that require the newer schema versions...
self._nwb_io.write(self.nwbfile, cache_spec=False)
self._nwb_io.close()
self._h5_file.close()

Expand Down
6 changes: 5 additions & 1 deletion ipfx/nwb_append.py
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,11 @@ def append_spike_times(input_nwb_path: PathLike,

nwbfile.add_processing_module(spike_module)

nwb_io.write(nwbfile)
# Because the NWB schema versions of NWB data produced by MIES are older
# we do not want to cache the newer schema versions that IPFX is currently using
# WARNING: Doing this may introduce fragility down the road though if IPFX writes NWB fields
# that require the newer schema versions...
nwb_io.write(nwbfile, cache_spec=False)
else:
raise ValueError("Cannot add spikes times to the nwb file: "
"spikes times already exist!")
Expand Down

0 comments on commit f86ba1c

Please sign in to comment.