Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HDF5 file structure reorganisation, and InterfaceComponent readability #312

Merged
merged 4 commits into from
May 23, 2023

Conversation

samcunliffe
Copy link
Member

This PR is a single git cherry-pick of the merge commit of #280 but now (eventually) targeting main.

Part of

Continues from

  1. Add dependence on libhdf5 and basic implementation class structure. #304
  2. Test-driven reading and writing basic datatypes to HDF5. #310
  3. Handle structs (except with complex data, that's nasty)  #311

Supercedes

Actual code notes and change description

... are in

For future working, I think the "move files" and "add tests" steps here should have been separate PRs.

  • Check the diff with files on main to ensure none of the review improvements are lost.

willGraham01 and others added 2 commits May 22, 2023 16:35
* Trying to decipher structs

* Further towards reading a struct

* Bypass the stack-smash bug.

MATLAB saves character arrays as uint16s, and HDF5 is unable to interpret them as chars. As such, we need to do manual conversion in the test.

* Rename binary and update contents
…ility (#280)

* Reorganise hdf5 unit tests

- Move uint16 to char/str functions to the unit test utils file
- Create hdf5_and_tdms_objects subdirectory of unit/ to hold unit tests on interaction between hdf5 and tdms classes
- Move the Matrix<double> test from test_hdf5_io into the new subdirectory
- Data files needed for unit tests are defined in a unit_test_utils namespace to avoid redefinition across multiple files

* Create file to test interface and hdf5 interactions

- Add docstrings to interface.h since these are missing and I've just had to work out what they do
- Create a matlab script that can reproduce the class_data.mat file which the hdf5 unit tests will try to create tdms objects from
- Create the barebones test_hdf5_interface file

* HDF5Reader can read from .mat file and produce an InterfaceComponent

* File restructure: accounting for how many tests we are going to have with HDF5

* Prune includes

* Add .mat file for HDF5-TDMS-object unit tests to run.

- Adds scripts to reproduce this data, so in theory a new user can run a short MATLAB script to reproduce this
- Had a play with trying to get setup-matlab to run these scripts before the unit tests, but alas, no.
@samcunliffe samcunliffe added the enhancement New feature or request label May 22, 2023
tdms/include/hdf5_io/hdf5_reader.h Show resolved Hide resolved
tdms/src/hdf5_io/hdf5_base.cpp Outdated Show resolved Hide resolved
tdms/src/hdf5_io/hdf5_base.cpp Outdated Show resolved Hide resolved
@codecov
Copy link

codecov bot commented May 22, 2023

Codecov Report

Patch coverage: 68% and no project coverage change.

Comparison is base (ac3d36d) 25% compared to head (658faac) 25%.

Additional details and impacted files
@@         Coverage Diff         @@
##           main   #312   +/-   ##
===================================
  Coverage    25%    25%           
===================================
  Files        63     66    +3     
  Lines      3596   3607   +11     
===================================
+ Hits        903    908    +5     
- Misses     2693   2699    +6     
Impacted Files Coverage Δ
tdms/include/hdf5_io/hdf5_reader.h 0% <0%> (ø)
tdms/src/hdf5_io/hdf5_base.cpp 96% <ø> (ø)
tdms/src/hdf5_io/hdf5_reader.cpp 100% <100%> (ø)
tdms/src/hdf5_io/hdf5_writer.cpp 100% <100%> (ø)

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@samcunliffe samcunliffe added the review required A review approval is required before merging label May 23, 2023
@samcunliffe samcunliffe marked this pull request as ready for review May 23, 2023 08:19
tdms/include/interface.h Outdated Show resolved Hide resolved
tdms/src/hdf5_io/hdf5_base.cpp Outdated Show resolved Hide resolved
// Read the InterfaceComponent in as a 2-element double array
double read_buffer[2];
read_field_from_struct("interface", plane, read_buffer);
// The index that is read in should have 1 subtracted from it, to account for
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Forward thinking here: if we/someone wants to also remove MATLAB from the input generation in the future, they'll need to develop some kind of workaround for these usage cases where we have to subtract one due to MATLAB indexing.

Base automatically changed from 181-cherry-pick-pr278 to main May 23, 2023 09:13
@samcunliffe samcunliffe added this pull request to the merge queue May 23, 2023
Merged via the queue into main with commit ec7a004 May 23, 2023
@samcunliffe samcunliffe deleted the 181-cherry-pick-pr280 branch May 23, 2023 12:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request review required A review approval is required before merging
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants