Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compressed Vol format #1228

Merged
merged 39 commits into from
Jan 20, 2017
Merged

Compressed Vol format #1228

merged 39 commits into from
Jan 20, 2017

Conversation

dcoeurjo
Copy link
Member

@dcoeurjo dcoeurjo commented Dec 27, 2016

PR Description

This PR adds a new Version (3) to Vol file format allowing to have compressed volumetric data using zlib. In Version 2, vol consists in an ASCII header and then the raw data in binary mode. This new version compresses the raw data using zlib/boost-filter, saving huge storage.

VolReader and VolWriter have been updated and can still import/export Version 2 vols.

To be able to perform such compression, zlib must be installed in the system (and is thus a new mandatory dependency in DGtal). This lib is highly standard and shouldn't be a problem. I could have make this dependency optional but if you all agree, I would like this "compressed" version to be the default one.

I've planned a upgradeVol tool to upgrade a vol file from Version 2 to 3 in DGtalTools.

Additionaly, the Boost lib filter I use needs boost>1.50.

🎄 🎁 🎄

TODO

  • update LongvolReader/LongvolWriter accordingly
  • Check DGtalConfig.cmake.in

Checklist

  • Unit-test of your feature with Catch.
  • Doxygen documentation of the code completed (classes, methods, types, members...)
  • Documentation module page added or updated.
  • New entry in the ChangeLog.md added.
  • No warning raised in Debug cmake mode (otherwise, Travis C.I. will fail).
  • All continuous integration tests pass (Travis & appveyor)

@dcoeurjo
Copy link
Member Author

this PR depends on #1227

@dcoeurjo
Copy link
Member Author

dcoeurjo commented Jan 3, 2017

@kerautret I think I would need your help to download and install zlib DLLs on appeveyor windows script.

The DLL is available here: http://www.winimage.com/zLibDll/ but I don't know of to deal with the appeveyor script.
thx

@kerautret
Copy link
Member

Nice features ! Yes I will look it !

phcerdan added a commit to phcerdan/DGtal that referenced this pull request Jan 11, 2017
Current behavior is to decompress tables at cmake level,
wasting HDD space of user.

Depends on PR DGtal-team#1228 that adds zlib dependency.
Discussion and motivation on Issue DGtal-team#1229.
@dcoeurjo
Copy link
Member Author

FYI: here you have a simple command line tool to convert compressed vol to raw https://github.com/dcoeurjo/VolGallery/blob/master/tools/vol2raw.sh

(yup, the compressed part can be decompressed using gzip with the appropriate header 😄 )

@dcoeurjo
Copy link
Member Author

The appveyor script is now ok (thanks @kerautret ). The DGtalTools target is still failing on travis but a DGtaltools pending PR has a fix.

Can someone have a look to the code so that I can merge this PR ? thx

phcerdan added a commit to phcerdan/DGtal that referenced this pull request Jan 19, 2017
Current behavior is to decompress tables at cmake level,
wasting HDD space of user.

Depends on PR DGtal-team#1228 that adds zlib dependency.
Discussion and motivation on Issue DGtal-team#1229.
@kerautret
Copy link
Member

yes I look it for the DGtalTools ;)

@kerautret
Copy link
Member

I reloaded Travis from the DGtools merged ;)

phcerdan added a commit to phcerdan/DGtal that referenced this pull request Jan 19, 2017
Current behavior is to decompress tables at cmake level,
wasting HDD space of user.

Depends on PR DGtal-team#1228 that adds zlib dependency.
Discussion and motivation on Issue DGtal-team#1229.
@kerautret
Copy link
Member

@dcoeurjo sorry I have included the master in the PR on your branch ;(

@dcoeurjo
Copy link
Member Author

no problemo;)

@dcoeurjo
Copy link
Member Author

Okay.. bots look fine. Merging this dude.
thanks everyone for your help

@dcoeurjo dcoeurjo merged commit 0988e28 into DGtal-team:master Jan 20, 2017
@dcoeurjo dcoeurjo deleted the CompressedVol branch October 7, 2021 11:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants