-
Notifications
You must be signed in to change notification settings - Fork 2.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Pipelined Implementation of ZSTD_dfast #2774
Merged
felixhandte
merged 17 commits into
facebook:dev
from
felixhandte:zstd-dfast-pipelined-single
Oct 13, 2021
Merged
Pipelined Implementation of ZSTD_dfast #2774
felixhandte
merged 17 commits into
facebook:dev
from
felixhandte:zstd-dfast-pipelined-single
Oct 13, 2021
Commits on Oct 5, 2021
-
Configuration menu - View commit details
-
Copy full SHA for 258c062 - Browse repository at this point
Copy the full SHA 258c062View commit details -
Configuration menu - View commit details
-
Copy full SHA for 1bdf041 - Browse repository at this point
Copy the full SHA 1bdf041View commit details -
Configuration menu - View commit details
-
Copy full SHA for 072ffaa - Browse repository at this point
Copy the full SHA 072ffaaView commit details -
Configuration menu - View commit details
-
Copy full SHA for a1ac720 - Browse repository at this point
Copy the full SHA a1ac720View commit details -
Hash Long One Position Ahead (+2.5% Speed)
Aside from maybe a latency win in the loop, this means that when we find a short match, we've already done the hash we need to check the next long match.
Configuration menu - View commit details
-
Copy full SHA for db4e1b5 - Browse repository at this point
Copy the full SHA db4e1b5View commit details -
Use Look-Ahead Hash for Next Long Check after Short Match (+0.5% Speed)
This costs a little ratio, unfortunately.
Configuration menu - View commit details
-
Copy full SHA for 39f2491 - Browse repository at this point
Copy the full SHA 39f2491View commit details -
Write Back Advanced Hash in Long Matches as Well (+Ratio)
Since we're now hashing the position ahead even if we find a long match and don't search that next position, we can write it back into the hashtable even in long matches. This seems to cost us no speed, and improves compression ratio slightly!
Configuration menu - View commit details
-
Copy full SHA for 2ddef7c - Browse repository at this point
Copy the full SHA 2ddef7cView commit details -
Advance Long Index Lookup (+0.5% Speed)
This lookup can be advanced to before the short match check because either way we will use it (in the next loop iter or in `_search_next_long`).
Configuration menu - View commit details
-
Copy full SHA for 6ae44c0 - Browse repository at this point
Copy the full SHA 6ae44c0View commit details -
Configuration menu - View commit details
-
Copy full SHA for 2cdfad5 - Browse repository at this point
Copy the full SHA 2cdfad5View commit details -
Configuration menu - View commit details
-
Copy full SHA for 47fd762 - Browse repository at this point
Copy the full SHA 47fd762View commit details -
Configuration menu - View commit details
-
Copy full SHA for fcab484 - Browse repository at this point
Copy the full SHA fcab484View commit details -
Configuration menu - View commit details
-
Copy full SHA for 051b473 - Browse repository at this point
Copy the full SHA 051b473View commit details -
Configuration menu - View commit details
-
Copy full SHA for 62536ef - Browse repository at this point
Copy the full SHA 62536efView commit details -
Configuration menu - View commit details
-
Copy full SHA for c2c3283 - Browse repository at this point
Copy the full SHA c2c3283View commit details -
This test depended on `_extDict` and `_noDict` compressing identically, which is not a guarantee we make, AFAIK.
Configuration menu - View commit details
-
Copy full SHA for 168d0a3 - Browse repository at this point
Copy the full SHA 168d0a3View commit details -
Configuration menu - View commit details
-
Copy full SHA for 79ca830 - Browse repository at this point
Copy the full SHA 79ca830View commit details
Commits on Oct 12, 2021
-
Configuration menu - View commit details
-
Copy full SHA for 0bfc935 - Browse repository at this point
Copy the full SHA 0bfc935View commit details
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.