Add virtual dataset support

[simonbyrne]

Adds an interface for supporting HDF5 virtual datasets.

[mkitti]

Could provide a way to set the virtual view property of the DatasetAccessPropertyList using h5p_set_virtual_view?

[mkitti]

Could you please rebase this?

[mkitti]

I'm also adding the cross reference to #617 and https://github.com/AStupidBear/HDF5Utils.jl/blob/master/src/virtual.jl as prior attempts to do this.

[mkitti]

Could we add a property API on DatasetAccessProperties via H5Pset_virtual_view() and H5Pget_virtual_view() for completeness?

[simonbyrne]

I'm also adding the cross reference to #617 and https://github.com/AStupidBear/HDF5Utils.jl/blob/master/src/virtual.jl as prior attempts to do this.

Ah, I wasn't aware of these. Thanks!

[mkitti]

Where are we on this?

[simonbyrne]

I've been pretty busy, so haven't had time to look at it further.

Are we happy with the interface? I was hoping to have a simpler higher level interface, but I guess it's okay for now.

It does need some docs as well.

Any other changes?

[mkitti]

I would focus on writing some minimal documentation and get this merged. I would prefer these pull requests not hang around for a long time. As noted previously, we've already been through several versions of this without it having been merged in.

The interface is pretty much as straightforward as it could be, so let's proceed with that.

[musm]

I'm getting the following error:

julia> d = create_dataset(
           f,
           "x",
           datatype(Float64),
           vspace;
           virtual=[HDF5.VirtualMapping(vspace, "./sub-%0b.hdf5", "x", srcspace)]
       )
ERROR: HDF5.API.H5Error: Error setting virtual
libhdf5 Stacktrace:
 [1] H5D_virtual_parse_source_name: Invalid arguments to routine/Bad value
     invalid format specifier
  ⋮
Stacktrace:
  [1] macro expansion
    @ C:\Users\Mus\.julia\packages\HDF5\5P16B\src\api\error.jl:18 [inlined]
  [2] h5p_set_virtual(dcpl_id::HDF5.DatasetCreateProperties, vspace_id::HDF5.Dataspace, src_file_name::String, src_dset_name::String, src_space_id::HDF5.Dataspace)
    @ HDF5.API C:\Users\Mus\.julia\packages\HDF5\5P16B\src\api\functions.jl:3960
  [3] push!
    @ C:\Users\Mus\.julia\packages\HDF5\5P16B\src\virtual.jl:43 [inlined]
  [4] append!
    @ C:\Users\Mus\.julia\packages\HDF5\5P16B\src\virtual.jl:50 [inlined]
  [5] set_virtual!
    @ C:\Users\Mus\.julia\packages\HDF5\5P16B\src\properties.jl:507 [inlined]
  [6] class_setproperty!(#unused#::Type{HDF5.DatasetCreateProperties}, p::HDF5.DatasetCreateProperties, name::Symbol, val::Vector{HDF5.VirtualMapping})
    @ HDF5 C:\Users\Mus\.julia\packages\HDF5\5P16B\src\properties.jl:550
  [7] setproperty!
    @ C:\Users\Mus\.julia\packages\HDF5\5P16B\src\properties.jl:59 [inlined]
  [8] (::HDF5.var"#4#5"{Tuple{HDF5.DatasetCreateProperties, HDF5.DatasetTransferProperties, HDF5.DatasetAccessProperties}})(::Pair{Symbol, Vector{HDF5.VirtualMapping}})
    @ HDF5 C:\Users\Mus\.julia\packages\HDF5\5P16B\src\properties.jl:139
  [9] filter
    @ .\abstractdict.jl:472 [inlined]
 [10] #setproperties!#3
    @ C:\Users\Mus\.julia\packages\HDF5\5P16B\src\properties.jl:135 [inlined]
 [11] create_dataset(parent::HDF5.File, path::String, dtype::HDF5.Datatype, dspace::HDF5.Dataspace; dcpl::HDF5.DatasetCreateProperties, dxpl::HDF5.DatasetTransferProperties, dapl::HDF5.DatasetAccessProperties, pv::Base.Pairs{Symbol, Vector{HDF5.VirtualMapping}, Tuple{Symbol}, NamedTuple{(:virtual,), Tuple{Vector{HDF5.VirtualMapping}}}})
    @ HDF5 C:\Users\Mus\.julia\packages\HDF5\5P16B\src\datasets.jl:57
 [12] top-level scope
    @ REPL[20]:1

[musm]

julia> versioninfo()
Julia Version 1.8.2
Commit 36034abf26 (2022-09-29 15:21 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: 16 × AMD Ryzen 9 5900HS with Radeon Graphics
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-13.0.1 (ORCJIT, znver3)
  Threads: 16 on 16 virtual cores
Environment:
  JULIA_EDITOR = code.cmd
  JULIA_NUM_THREADS = auto

[mkitti]

Oops, we did not add virtual_dataset.jl to runtests.jl

julia> HDF5.API.SHORT_ERROR[] = false
false

julia> d = create_dataset(
           f,
           "x",
           datatype(Float64),
           vspace;
           virtual=[HDF5.VirtualMapping(vspace, "./sub-%0b.hdf5", "x", srcspace)]
       )
ERROR: HDF5.API.H5Error: Error setting virtual
libhdf5 Stacktrace:
 [1] H5D_virtual_parse_source_name: Invalid arguments to routine/Bad value
     invalid format specifier
 [2] H5Pset_virtual: Property lists/Unable to initialize object
     can't parse source file name

[simonbyrne]

Ah, it should be just %b: the docs are wrong, they never actually added support for the dimension specifier. I'll open a PR