add grep-cli crate #1038
Merged
add grep-cli crate #1038
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This commit moves a lot of "utility" code from ripgrep core into grep-cli. Any one of these things might not be worth creating a new crate, but combining everything together results in a fair number of a convenience routines that make up a decent sized crate. There is potentially more we could move into the crate, but much of what remains in ripgrep core is almost entirely dealing with the number of flags we support. In the course of doing moving things to the grep-cli crate, we clean up a lot of gunk and improve failure modes in a number of cases. In particular, we've fixed a bug where other processes could deadlock if they write too much to stderr. Fixes #990
These flags provide granular control over ripgrep's buffering strategy. The --line-buffered flag can be genuinely useful in certain types of shell pipelines. The --block-buffered flag has a murkier use case, but we add it for completeness.
The --pre-glob flag is like the --glob flag, except it applies to filtering files through the preprocessor instead of for search. This makes it possible to apply the preprocessor to only a small subset of files, which can greatly reduce the process overhead of using a preprocessor when searching large directories.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR rounds out the initial functionality of libripgrep by pushing several pieces of somewhat CLI specific functionality into a utility library. This library, called
grep-cli, includes things like reading process output (which also fixes #990), decompressing files, unescaping text and reading patterns from files. Throughout most operations, we retain good failure modes by giving descriptive error messages.We also add a couple new minor flags:
--line-bufferedand--block-bufferedfor granular control over ripgrep's buffering strategy. (GNU grep has a--line-bufferedflag as well.)--pre-globfor filtering files sent through the preprocessor specified by--pre. This makes it plausible to use--preon large directories if it is only actually needed for a small subset of files. (Otherwise, the process overhead tends to make the search take unbearably long.)