Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Searching filenames (-g option in ag) #91

Closed
Valloric opened this issue Sep 25, 2016 · 63 comments
Closed

Searching filenames (-g option in ag) #91

Valloric opened this issue Sep 25, 2016 · 63 comments

Comments

@Valloric
Copy link

Valloric commented Sep 25, 2016

Ag supports using -g to recursively search filenames matching the provided pattern. Ripgrep doesn't have this feature (to the best of my knowledge; I've read the --help output three times looking for it); it's the only ag feature holding me back from fully switching to ripgrep. :)

Since ripgrep already has something unrelated mapped to -g, the option name should be something else. The incompatibility with ag will be unfortunate, but survivable.

@BurntSushi
Copy link
Owner

I'm pretty sure this has been reported at least several times now. :-) rg has --files for listing files and -g for glob matching.

@Valloric
Copy link
Author

I'm pretty sure this has been reported at least several times now. :-)

Sorry about that! It might be indicative of a documentation "bug"; it seems people are having trouble locating this feature.

@Valloric
Copy link
Author

Valloric commented Sep 25, 2016

Ah, I get it now. It's a combination of two flags: --files lists all the files that would have been searched and -g can be used to only search files matching a pattern.

This is very non-obvious. It requires users to fully understand two separate, unrelated options and to realize that when combined, they can be used to solve their problem.

Since ripgrep will have lots of users coming from ag who will expect a simple single-flag solution, it might be a good idea to have a single flag (much like ag) for this use-case, if for any reason, than to save you the headache of closing all these issue reports. :)

@BurntSushi
Copy link
Owner

I'd like to solve this with better documentation. It's clear this is an important feature to many, but it's decidedly subservient to ripgrep's primary focus, so I'd like to avoid adding extra flags for it.

@gabrielmagno
Copy link

gabrielmagno commented Sep 29, 2016

That would be a great feature. But it is important to notice the differences.

Let's say I have this directory tree:

.
└── path
    └── to
        ├── file
        │   ├── pattern (file)
        │   └── pattern.txt (file)
        ├── pattern
        │   └── fileA.txt (file)
        └── patterns
            └── fileB.txt (file)
  • ag returns all the files that have "pattern" in any part of its path
$ ag -g "pattern"
path/to/pattern/fileA.txt
path/to/patterns/fileB.txt
path/to/file/pattern
path/to/file/pattern.txt
  • rg returns all the files that have "pattern" as its name
$ rg --files -g "pattern"
path/to/file/pattern

So, ideally, if such a feature is implemented in ripgrep to reproduce what The Silver Search does, it should check the path (not only the name) and also match any part of the string (not an exact match, not a glob).

@BurntSushi
Copy link
Owner

BurntSushi commented Sep 29, 2016

Remember that -g gives you the ability to use a glob, so if you want to match any component of the path, then **/pattern/** will work. (Well, to match pattern.txt and path/to/patterns, you will actually need **/pattern*/**.)

@nateozem
Copy link

In case somebody wouldn't know the syntax for doing a search with the condition of excluding files with a certain name using the command line, here is an example to do so:

$ rg "from_str" -g "\!tags" -g "repos/*"
                    ^^^

As shown, you may need to escape ! with \, if wish to exclude files from search.

@BurntSushi
Copy link
Owner

BurntSushi commented Dec 27, 2016 via email

@h5rdly
Copy link

h5rdly commented Dec 30, 2016

Are there search parameters for rg that would list any files that match pattern, but any folder that matches pattern will only appear by itself, aka not relisted with everything inside it?

Say, for 'python' I'd like to get -
c:/python27/
c:/python27/python.exe
c:/python27/pythonw.exe
c:/pdfs/pythonconf.pdf
.[more such stuff]
.

Is that possible directly, without parsing the output?

@BurntSushi
Copy link
Owner

No. ripgrep will never print directories, only file paths. ripgrep isn't a find replacement.

Note that there's enough of libripgrep done that you could write your own CLI tool in Rust fairly easily to do this. I would be happy to mentor that project.

@blueyed
Copy link
Contributor

blueyed commented Jan 21, 2017

Note that --files also includes binary files (is this a bug?)
The help says:

Print each file that would be searched without actually performing the search.

So to replicate ag -g . I am using rg --files-with-matches . now.
A difference here is that rg does not include empty files like ag does, even when using '' or '.?' for the pattern.

@BurntSushi
Copy link
Owner

Note that --files also includes binary files (is this a bug?)

No. "binary" files isn't a fact one can know without actually searching the file contents.

So to replicate ag -g . I am using rg --files-with-matches .

Those aren't the same. ag -g . is like rg --files. rg --files-with-matches is like ag --files-with-matches.

A difference here is that rg does not include empty files like ag does, even when using '' or '.?' for the pattern.

Whether a file is empty or not has no impact on the output of --files. In an empty directory:

$ touch hi
$ rg --files
hi

@blueyed
Copy link
Contributor

blueyed commented Jan 21, 2017

@BurntSushi
Thanks for your reply.
I got confused because I had a global .agignore file that excluded *.png, and thought that ag -g . would exclude binary files therefore already.

Whether a file is empty or not has no impact on the output of --files

I was referring to --files-with-matches here, which I think is better to exclude binary files (I am using rg here to list files for ctags).
I think it's good to not include empty files here, but still wonder if you could have a pattern that is always true, also for empty files (e.g. ^ or .?)?

@BurntSushi
Copy link
Owner

BurntSushi commented Jan 21, 2017

I was referring to --files-with-matches here, which I think is better to exclude binary files (I am using rg here to list files for ctags).

It does:

$ echo -e 'z\x00' > test
$ rg --files-with-matches z
$ rg --files-with-matches -a z
test

Please note that "binary" detection cannot possibly be perfect. It uses a simple heuristic: if there's a NUL byte, then the file is considered binary. This is what GNU grep does. (ripgrep does have a bug here, see #52.)

I think it's good to not include empty files here, but still wonder if you could have a pattern that is always true, also for empty files (e.g. ^ or .?)?

ripgrep is a line oriented searcher. If a file doesn't contain any lines, then there's nothing to match. What is your use case?

In the future, could you please open new issues for new bug reports/feature requests?

@blueyed
Copy link
Contributor

blueyed commented Jan 21, 2017

I was referring to --files-with-matches here, which I think is better to exclude binary files (I am using rg here to list files for ctags).

It does:

And that is good.

ripgrep is a line oriented searcher. If a file doesn't contain any lines, then there's nothing to match.

I see, but somehow thought that ^ should be always true.

But grep -l '^' * behaves the same, it makes sense, and it is a good way to skip empty files (using ., which would work even when ^ would stand for "anything").

In the future, could you please open new issues for new bug reports/feature requests?

I get you, but it is still about mimicking ag -g (and ag -g . in particular), isn't it?

What is your use case?

(I am using rg here to list files for ctags).

I was using ag -g . | ctags --links=no -L- to generate tags for my dotfiles (a globally included tags file).

I've replaced it now with rg --files-with-matches --no-ignore-vcs . | ctags --links=no -L-.

@BurntSushi
Copy link
Owner

I get you, but it is still about mimicking ag -g (and ag -g . in particular), isn't it?

Definitely not. I think there is a lot of value in having a similar interface as other tools, but I've never added something to ripgrep just for the sake of mimicking another tool. :-)

In the future, please open new issues to discuss new features or bugs. This particular issue is closed and done with. If there's a problem, we should start a new discussion.

I've replaced it now with rg --files-with-matches --no-ignore-vcs . | ctags --links=no -L-.

Why does this require printing empty files? Surely you won't be able to jump to any symbol defined in an empty file. ;-)

@blueyed
Copy link
Contributor

blueyed commented Jan 21, 2017

@BurntSushi
I do not want the empty files - it's a nice side effect of the mimicking / workaround though.

about mimicking ag -g (and ag -g . in particular), isn't it?

Definitely not. I think there is a lot of value in having a similar interface as other tools, but I've never added something to ripgrep just for the sake of mimicking another tool. :-)

Sure, but then this issue is still the place to come to when you want to simulate it - that's what I've meant.

All is fine from my side.. :)
Thanks!

@BurntSushi
Copy link
Owner

@blueyed I see, OK. I had a really hard time understanding what you were trying to say! Glad all is well.

@blueyed
Copy link
Contributor

blueyed commented Jan 21, 2017

@BurntSushi
Thanks for a faster and bug-free (with regard to the .gitignore handling) replacement for ag!

@Valloric
Copy link
Author

Valloric commented Mar 2, 2017

Seems like I misunderstood what -g did originally; it's for a glob match, not a regex match.

Given that, I can't say that rg --files -g 'pattern' is adequate. What I'd like to see as a user is a regex match on the filenames.

@mcandre
Copy link

mcandre commented Aug 16, 2017

Is there no combination of flags for ripgrep allowing users to constrain both the filename and the content simultaneously? I'd like to search for Makefile's with "$$" in them, for example. Currently working around this with find . -name Makefile -exec grep "\\$\\$" {} \;

@BurntSushi
Copy link
Owner

@mcandre What have you tried? If rg -F '$$' searches too much, then restrict it using exactly the techniques outlined above: rg -F '$$' -g Makefile.

@mcandre
Copy link

mcandre commented Aug 16, 2017

@BurntSushi Thanks for the tip, works like a charm!

I wonder why the ripgrep CLI parsing doesn't treat

rg -g Makefile '$$'

as

rg -g Makefile -F '$$'

Oh well, at least the latter works on my machine^TM.

@okdana
Copy link
Contributor

okdana commented Aug 16, 2017

Because $ is a regex meta-character used to match the end of the line; -F escapes all meta-characters so that they're treated literally, which is what you wanted in this case

Might be worth reviewing something like http://www.regular-expressions.info/quickstart.html

@mcandre
Copy link

mcandre commented Aug 23, 2017

@okdana Lol I totally forgot about that! Yeah, -F '$$' works on my machine, and I suppose '\$\$' would work as well.

@rosshadden
Copy link

@BurntSushi I apologize if you find these kinds of issues annoying, but clearly based on your comments in some of them you understand that a lot of us want this behavior. Your focus up to this point has been on showing that ripgrep already has the behavior, but I think you should heavily consider focusing on making it as easy as it is in ag, pt, and ack.

Using -g in all of those tools is not just a gimmick, it's incredibly useful. In fact I use it just about as much as I do the main functionality. A lot of projects like fzf, unite.vim, and denite.vim even officially recommend using this feature for getting lists of files, as -g '' is a really convenient and fast git-agnostic way to do git ls-files. In fact doing a search for " -g" on fzf's readme shows five occurrences of using ag -g in different ways.

To be clear I'm not proposing you make it -g. But please consider adding a flag that does the equivalent. It will shut all of us up and make us feel much better about jumping into using and loving ripgrep. Thanks!

@BurntSushi
Copy link
Owner

@rosshadden What are you actually asking for? The -g flag's behavior is already set and it isn't changing. Are you asking for a new flag that combines -g and --files? If so, just define an alias?

alias rgg="rg --files"

@borekb
Copy link

borekb commented Aug 20, 2019

Thanks for the mention of fd! I cloned a repository somewhere on my disk and wanted to abuse ripgrep to find it; fd is the right tool for that as I need to list only directories.

@WhyNotHugo
Copy link

I'm trying to achieve what's been discussed on this thread, but it's not working for me, I'm not sure if I've misunderstood something, or if something's broken. Say I want to find files named vendor:

$ rg --files . -g vendor
$ rg --files vendor
vendor: No such file or directory (os error 2)
$ find . -iname vendor
./idf/core/static/www/js/vendor
./idf/core/static/www/css/vendor
./enviatufoto/static/www/js/vendor

What am I doing wrong? With ag, this is basically ag -g vendor, but I can't figure out how it works with rg.

@BurntSushi
Copy link
Owner

@WhyNotHugo The glob is applied to every path ripgrep traverses. Is vendor a directory? If so, vendor on its own won't match anything inside of vendor. Use rg --files -g '**/vendor/**' instead.

@lamyergeier
Copy link

lamyergeier commented Jun 29, 2020

@BurntSushi --files -g doesn't respect the contents of .gitignore

rg --files --type md -g "*Python*" "${PWD}"

@BurntSushi
Copy link
Owner

BurntSushi commented Jun 29, 2020

Please file a new and complete big report. Your comment isn't actionable.

@lamyergeier
Copy link

lamyergeier commented Jun 29, 2020

I solved this as follows:

To search Python in filenames:

Search=Python
rg --files "${PWD}" | rg --regexp "${Search}[^/]*$" | sort | nl

@jjjchens235
Copy link

jjjchens235 commented Oct 6, 2020

piggybacking off of @anishmittal2020
This function returns any file name AND directory matches, though directories themselves cannot be returned, as the --file flag returns a list of files only, directory paths are excluded.

function f() {
	#find any files or directories that match arg
	rg --files "${PWD}" | rg --regexp ".*/.*$1.*" #| sort | nl
}

Put in .bashrc, and then call it on the command line.
If searching for filename or directory names that contain the word 'vendor':
f vendor

Taking @WhyNotHugo example:

./idf/core/static/www/js/vendor/bar.py
./idf/vendor.py
./idf/core/static/www/js/vendor/

The first two files will be returned, the first because the path contains a dir called vendor, and the second because it contains a filename with vendor. The last file is a dir, it will not be returned.

Edit: I hate to admit it, but I should have just used fd from the get-go. It allows for regex arguments, and user can specify if they want to search for files, directories, or both.

function f() {
	#list of flags: https://github.com/sharkdp/fd#command-line-options
	#optional flags should be passed in first, then PATTERN and PATH
	fd -H -a -p  "$@"
}

@AtomicNess123
Copy link

AtomicNess123 commented Feb 7, 2021

I have read the whole thread. I have not successfully managed to just find all filenames containing "helm".
When I do:

rga ~/.emacs.d --files -g "*helm*"

I just finds filenames with detached "helm" (i.e., "xxx-helm-xxx.el"), but not "xxxhelmxxx.el".
How to specify this? Thanks!

@BurntSushi
Copy link
Owner

@AtomicNess123 Works just fine for me:

$ touch xxx-helm-xxx.el xxxhelmxxx.el
$ l
total 0
-rw-rw-r-- 1 andrew users 0 Feb  7 11:40 xxx-helm-xxx.el
-rw-rw-r-- 1 andrew users 0 Feb  7 11:40 xxxhelmxxx.el
$ rg --files -g '*helm*'
xxx-helm-xxx.el
xxxhelmxxx.el

@AtomicNess123
Copy link

Thanks. Actually, it does work in your example.
But not when I do:

§ rg --files -g '*helm*' ~/.emacs.d

In this case, when I specify the folder to search within, it only finds the "xxx-helm-xxx.el" instances, and not the "xxxhelmxxx.el" ones.

@BurntSushi
Copy link
Owner

Also works just fine:

$ mkdir .emacs.d
$ touch .emacs.d/xxx-helm-xxx.el .emacs.d/xxxhelmxxx.el
$ rg --files -g '*helm*' .emacs.d/
.emacs.d/xxx-helm-xxx.el
.emacs.d/xxxhelmxxx.el

I can't think of a reason why it isn't working for you. Sorry. To be honest, it doesn't make sense to me why *helm* would match xxxhelmxxx but not xxx-helm-xxx. I suspect there is something else going awry. Please consider making a smaller reproduction.

@AtomicNess123
Copy link

I won't work like this in my system.

rg --version  
ripgrep 12.1.1

What is your version?

@BurntSushi
Copy link
Owner

Same, but there aren't any recent changes that would impact this AFAIK. Please consider trying my exact set of commands in a fresh directory. Try running with the --debug flag. Paste its output and all the exact commands you're running.

@AtomicNess123
Copy link

Other question: how many levels (subdirectories) will the command search?

@BurntSushi
Copy link
Owner

All...

@AtomicNess123
Copy link

Then, it makes no sense. It works in your example. But I want to find any filename with "helm" in the name. If I search from root or any other folder, it only finds the files at the .emacs.d/modules and .emacs.d/other, but not in .emacs.d/elpa. It is very strange. But when I enter the elpa directory and run the command from there, it finds all the files.

@AtomicNess123
Copy link

Ok... does .gitignore affect results? It seems elpa is in .gitignore!

@BurntSushi
Copy link
Owner

Of course. The first two sentences of the readme:

ripgrep is a line-oriented search tool that recursively searches your current directory for a regex pattern. By default, ripgrep will respect your .gitignore and automatically skip hidden files/directories and binary files.

This is why I suggested the --debug flag. It will tell you which things ripgrep skips and why.

@AtomicNess123
Copy link

Yes, you were right. I'm just newbie in command line functions. I wonder if it's possible to override .gitignore setting?

@BurntSushi
Copy link
Owner

Sure. See the guide. If you have a specific question then open a new discussion post.

@sergeevabc
Copy link

sergeevabc commented May 2, 2021

Sorry to dig up this issue once again, but as a Windows user I believe workarounds provided earlier are not sterling, because alias rgg that combines --files -g part-of-filename is a Linux thing and fd utility to use instead of rg to search for filenames is still buggy to be relied upon. Yet this task arises daily (at least on my end), so I would vote with both hands for a short switch (to remember it easily and not to type long now imaginary --search-for-filenames). For example, -ff.

@BurntSushi
Copy link
Owner

Other software being buggy is not something I'm receptive towards as a motivation for adding more features to ripgrep.

@AtomicNess123
Copy link

piggybacking off of @anishmittal2020
This function returns any file name AND directory matches, though directories themselves cannot be returned, as the --file flag returns a list of files only, directory paths are excluded.

function f() {
	#find any files or directories that match arg
	rg --files "${PWD}" | rg --regexp ".*/.*$1.*" #| sort | nl
}

Thanks for this. What is the meaning of "${PWD}"?

@chris-w-jarvis
Copy link

Whats wrong with doing rg --files | rg foo ?

This seems to work fine for me :)

@foucist
Copy link

foucist commented Mar 23, 2022

Inspired by the earlier alias, I'm trying something like this. Function is named ag because old habits die hard 😅

function ag() { rg --files | rg ".*/.*$1.*" | sort | rg $1 }

@mcandre
Copy link

mcandre commented Mar 24, 2022 via email

mabo3n added a commit to mabo3n/.spacemacs.d that referenced this issue Jun 21, 2022
This makes it possible to preprocess query arguments on the fly.

One use case (which is already implemented in this commit) is
the transformation of `-Gpattern` to `--iglob=**/*pattern*/**`,
which makes it possible to easily restrict rg searches to files
containing `pattern` in the path.

ag's -g flag probably will never be implemented in rg:
BurntSushi/ripgrep#91.
@PyKoch
Copy link

PyKoch commented May 23, 2024

In case somebody wouldn't know the syntax for doing a search with the condition of excluding files with a certain name using the command line, here is an example to do so:

$ rg "from_str" -g "\!tags" -g "repos/*"
                    ^^^

As shown, you may need to escape ! with \, if wish to exclude files from search.

Please excuse the newbie question here:
I am trying to search all pdf files and want to exclude those that contain 'ms' in the file name.
This is what I entered:
rg --type pdf -i "velocity" --sort-files -g '\!ms' -g '*paper_3*' >> test_p3_no_ms.txt
Sadly, there are still files with 'ms' in the title included. Any help would be greatly appreciated.

@BurntSushi
Copy link
Owner

@PyKoch Please don't ask questions in old unrelated issues. This issue is a feature request for some other feature.

And when you do ask questions, please include a reproduction. Here's how you do it: you include all relevant details so that others can see what you're seeing. This means providing inputs, showing the outputs you see and the outputs you want to see. Basically, it comes down to "show, don't tell." Here, I'll show you.

First create an empty directory with some files:

$ mkdir rg-search-issue
$ cd rg-search-issue
$ touch msfoo fooms ms quux

And now run rg --files to show that it correctly finds all of the files I created above:

$ rg --files
quux
ms
fooms
msfoo

And now my attempt at filtering:

$ rg --files -g '\!ms'
$

If you did that, then folks helping you wouldn't have to guess at your problem.

The first thing to realize here is that ripgrep isn't printing any files. This seems different from what you report where there are files being printed. Since you didn't share any details about your environment, and you provided a command more complex than necessary to demonstrate your specific issue with filtering out things that contain ms, I'm not sure what's going on.

In any case, there are two issues here. Assuming you're on Unix, the first issue is that you're escaping ! in a single quoted value. A comment above said that ! needed to be escaped, but that was in a double quoted string. The second issue is that the glob ms doesn't match AAmsAA. The glob ms only matches ms. You need a wildcard to match around it. You seemingly did this for *paper_3*... So just do it for ms:

$ rg --files -g '!ms'
quux
fooms
msfoo
$ rg --files -g '!*ms*'
quux

In the future, please don't ask questions in old issues. There's an entire discussion forum open for that purpose.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests