Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve compile speed and binary size in opt #2763

Merged
merged 1 commit into from
Sep 2, 2021

Conversation

senhuang42
Copy link
Contributor

@senhuang42 senhuang42 commented Sep 1, 2021

Remove basically all inlining/templating in opt.
I measure no impact to end-to-end compression speed at all (levels 13-16, silesia.tar).

macOS with gcc

dev:
zstdcli binary size: 1703976
time to compile opt.o: ~8.5s

opt_compiletime:
zstdcli binary size: 1458352  (14.5% reduction)
time to compile opt.o: ~1.8s

The bigger bottleneck is still lazy, which will require a bit more work to reduce without speed regressions.

@senhuang42 senhuang42 merged commit 71076b7 into facebook:dev Sep 2, 2021
@Cyan4973
Copy link
Contributor

Cyan4973 commented Sep 8, 2021

I measure no impact to end-to-end compression speed at all (levels 13-16, silesia.tar).

I checked that statement.
To begin with, zstd_opt only starts at level 16 for large files, so obviously levels 13-15 will remain identical after this change.

Then, from level 16 to 22, I measured a -5 to -10% speed drop across a few scenarios on my Ubuntu station (using gcc-9.3).

I also measured the mentioned binary size reduction, as well as the sharp compilation time reduction.

So now it's a matter of trade-off, since it's not completely free.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants