cannot save custom type data

[milktrader]

julia> using MarketData

julia> cl
500x1 TimeArray{Float64,1} 2000-01-03 to 2001-12-31

             Close
2000-01-03 | 111.94
2000-01-04 | 102.5
2000-01-05 | 104.0
2000-01-06 | 95.0
⋮
2001-12-26 | 21.49
2001-12-27 | 22.07
2001-12-28 | 22.43
2001-12-31 | 21.9


julia> @save "cl.jld" cl
ERROR: This is the write function for CompositeKind, but the input is of type Date{ISOCalendar}
 in write_composite at /Users/Administrator/.julia/v0.3/HDF5/src/jld.jl:644
 in write at /Users/Administrator/.julia/v0.3/HDF5/src/jld.jl:577
 in write at /Users/Administrator/.julia/v0.3/HDF5/src/jld.jl:599
 in write at /Users/Administrator/.julia/v0.3/HDF5/src/jld.jl:577
 in write_composite at /Users/Administrator/.julia/v0.3/HDF5/src/jld.jl:686
 in anonymous at no file

[simonster]

Date is a custom bitstype. We can and should probably support this, but it's pretty uncommon.

[milktrader]

Ah, ok. So this is not supported presently. The motivation is to compress the datasets in the MarketData package so it's not parsed from a csv file every time the package is opened. I'll look into how RDatasets handles this for some clues

[milktrader]

I was looking for something similar to save() and load() as R handles this to create .Rda objects. Is this not the correct package for doing something like this?

[simonster]

No, this is the right package. JLD works fine with built-in numeric types, regular custom types, and immutables. We just do (yet) have support for custom bitstypes (i.e. types declared as bitstype 64 Date). HDF5 supports this, so we can implement it, but I didn't even think of these types of types as something that we needed to support until you pointed out that the Datetime package uses this.

[milktrader]

Ok, great. Right now the MarketData package takes quite a bit of time to load (about 15-18 seconds) and that's with only 3 large datasets. I'm planning to add a couple dozen more but need a faster load time solution.

[timholy]

Fixed

[milktrader]

Thanks Tim!

[simonster]

@timholy We should actually be able to save custom bitstypes of any size as opaque user-defined HDF5 types (see the HDF5 manual section 6.4.3.1). I will revisit this when I implement #27, which at this point I have neglected for far too long.