compression: add support for the zstd algorithm #363
Conversation
|
PR for c/image: containers/image#639 |
|
Modifying |
I've added the dependency, the generated |
giuseppe
changed the title
giuseppe
force-pushed
the
zstd
branch
4 times, most recently
from
672e3e9
to
61bf0da
Compare
giuseppe
changed the title
|
ready for review |
pkg/archive/archive_cgo.go
Outdated
| "github.com/DataDog/zstd" | ||
| ) | ||
|
|
||
| func zstdReader(buf *bufio.Reader) (io.ReadCloser, error) { |
There was a problem hiding this comment.
Why does this expect a *bufio.Reader instead of an io.Reader?
There was a problem hiding this comment.
I had left that as with unbuffered input I see very bad compression results.
But I guess it is better to be more generic, I've changed it to io.Reader.
pkg/archive/archive_nocgo.go
Outdated
| "io" | ||
| ) | ||
|
|
||
| func zstdReader(buf *bufio.Reader) (io.ReadCloser, error) { |
There was a problem hiding this comment.
Why does this expect a *bufio.Reader instead of an io.Reader?
zstd is a compression algorithm that has a very fast decoder, while providing also good compression ratios. The fast decoder makes it suitable for container images, as decompressing the tarballs is a very expensive operation. This is a first step at supporting zstd as we We don't yet generate zstd layers. In my testing, copying the Fedora image from a local dir: repository, the wall clock time passed from ~8s with gzip compression to ~4.5s with zstd. Signed-off-by: Giuseppe Scrivano <[email protected]>
|
LGTM |
1 similar comment
|
LGTM |
zstd is a compression algorithm that has a very fast decoder, while
providing also good compression ratios. The fast decoder makes it
suitable for container images, as decompressing the tarballs is a very
expensive operation. This is a first step at supporting zstd as we We
don't yet generate zstd layers.
In my testing, copying the Fedora image from a local dir: repository,
the wall clock time passed from ~8s with gzip compression to ~4.5s
with zstd.
Signed-off-by: Giuseppe Scrivano [email protected]