Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

caom2-pkg-service: Allow files larger than 8GB #108

Open
at88mph opened this issue Apr 30, 2021 · 0 comments
Open

caom2-pkg-service: Allow files larger than 8GB #108

at88mph opened this issue Apr 30, 2021 · 0 comments

Comments

@at88mph
Copy link
Member

at88mph commented Apr 30, 2021

Here is a search for a SITELLE observation:

https://www.cadc-ccda.hia-iha.nrc-cnrc.gc.ca/en/search/?Observation.observationID=2384125p&Observation.proposal.id=19AC27&[email protected]=NONE&Plane.position.bounds=NGC4569&Observation.collection=CFHT&Observation.instrument.name=SITELLE&Plane.energy.bandpassName=SN3#sortCol=caom2%3APlane.time.bounds.lower&sortDir=dsc&col_1=_checkbox_selector;;;&col_2=caom2%3AObservation.uri;;;&col_3=caom2%3AObservation.collection;;;&col_4=caom2%3AObservation.observationID;;;&col_5=caom2%3AObservation.sequenceNumber;;;&col_6=caom2%3APlane.productID;;;&col_7=caom2%3APlane.position.bounds.cval1;;;&col_8=caom2%3APlane.position.bounds.cval2;;;&col_9=caom2%3AObservation.target.name;;;&col_10=caom2%3APlane.time.bounds.lower;;;&col_11=caom2%3APlane.time.exposure;;;&col_12=caom2%3AObservation.instrument.name;;;&col_13=caom2%3APlane.energy.bandpassName;;;&col_14=caom2%3APlane.calibrationLevel;;;&col_15=caom2%3AObservation.type;;;&col_16=caom2%3AObservation.proposal.id;;;&col_17=caom2%3AObservation.proposal.pi;;;&col_18=caom2%3APlane.dataRelease;;;

Using the package service endpoint (https://www.cadc-ccda.hia-iha.nrc-cnrc.gc.ca/caom2ops/pkg?ID=ivo%3A%2F%2Fcadc.nrc.ca%2FCFHT%3F2384125p%2F2384125p) will result in a file containing an error message, but it has a .tar extension:

unexpected failure: java.lang.RuntimeException: entry size '8606365612' is too big ( > 8589934591 ).
CFHT-2384125p-2384125p/README0100644 0000000 0000000 00000000222 14043065762 013230 0ustar000000000 0000000 OK 2384125z.hdf5 1ca4483aac2b46e6281a80d41b36aa39 https://www.cadc-ccda.hia-iha.nrc-cnrc.gc.ca/data/pub/CFHT/2384125z.hdf5

As the HD5 file is larger than 8GB.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant