live_grep_raw

[jesseleite]

I proposed this idea on the Telescope Gitter, and there seemed to be interest from fearless leader @tjdevries, among others. The idea is to port vim-agriculture's functionality into a live_grep_raw Telescope finder.

Behaviour

• It would behave like the live_grep finder, but would not escape user input, allowing regex search by default:
  • func.*someFunc
• It would also allow the user to pass CLI flags and/or a path argument, right into the Telescope prompt:
  • -u "func.*someFunc" vendor
• If the command returns without errors, show a typical result count (ie. 2 / 6 or 0 / 0) at the right side of the prompt.
• We should gracefully handle errors (due to incomplete regex, etc):
  • Replace the 0 / 0 result count with an Error string
  • Return all of rg's error output in the preview window
• We should set default max result count to 1000 or something, to minimize chaos.

Example

Todo

• Support both rg and ag by default.
  • Telescope already uses rg by default.
  • Configure vimgrep_arguments = {'ag', '--nogroup', '--column'} to use ag.
• Accept params before a quoted query string.
• Accept params after a quoted query string.
• Accept path(s) after a quoted query string.
• Highlight/colour matching query output on each result line.
  • Build custom sorter that highlights regex?
  • Or find a way to render highlights from rg directly?
• Gracefully handle errors.
• Handle max result chaos.
  • @tjdevries said he might already have something for limiting max results?

[Conni2461]

@jesseleite do you need any help here?

[jesseleite]

Not ignoring Conni here! We chatted in gitter. I just need to find the time to dig in again, will do soon 😅

[muniter]

Highlight/colour matching query output on each result line.

Ripgrep has the '--json' option which returns the matches to be highlighted. Might be very useful If you haven't consider it. @jesseleite

https://www.mankier.com/1/rg#--json

[jesseleite]

@muniter Oh interesting, will check it out thanks!

[Shatur]

@jesseleite, are you planning to continue working on it?

[jesseleite]

@Shatur I am, but we just had a baby, and @tjdevries told me it might be good to wait for all the async stuff he's working on to be merged in.

[jesseleite]

PS. I got asyncing feeling that some people will misinterpret that last comment. @tjdevries and I dadn't have a baby; My wife and I had a baby.

[TC72]

Hey @jesseleite, I've had some time to look at this and it's great.
Just wanted to check what async features you're waiting for from @tjdevries?

Your code is using finders.new_job which has now been changed internally to use the new async_job_finder.
So I think your code is good to go on that front.

[weeman1337]

Because I really need the option passing parameters to grep I created an extension based on this PR:

https://github.com/nvim-telescope/telescope-live-grep-raw.nvim

@tjdevries as mentioned in #1228 telescope should be separated into smaller pieces for a better workload balance of the contributors. I would be happy moving this plugin to the nvim-telescope organisation and would also apply as maintainer ;)

[Conni2461]

@weeman1337 I can also move your project into this organization

[weeman1337]

@Conni2461 cool ✓ I will directly close my project afterwards

[Shatur]

@weeman1337 are you planning to move your project to the organization?

[weeman1337]

That is my plan. But I cannot do that on my own ;)

[Conni2461]

If you give me access to your repo i can do it. Or i create your a new repo here and i give you access and you push your repo to that remote. Whatever you prefer 😆

[weeman1337]

The new extension has been moved to the Telescope project:
https://github.com/nvim-telescope/telescope-live-grep-raw.nvim 🥳

[tjdevries]

So can we close this PR then?

[jesseleite]

I apologize for letting this go so long. Just trying the extension now, very cool! If @weeman1337 is up for maintaining this as an extension, I'm cool with closing my PR 👍

That said, the extension doesn't quite work to the spec I was going for. A couple things I noticed off the bat...

• I can't search a string like function create without wrapping it in quotes (ie. 'function create').
  • I personally think spaces should be treated as part of the search query, unless command line options are needed, or quotes are part of the query you are trying to search.
• The highlighting only works for literal single-word queries, but not when quoting or performing a regex query.
  • My idea was to either parse rg's --json output, or parse the ansi colour codes in rg's output, to determine how to highlight the telescope results list. If rg itself can highlight complex patterns, we should let it do the dirty work, and extract our highlighting from its algorithm imho.
  • I can't remember @TC72, but I believe you might have worked some of this out?

Thoughts?

[tjdevries]

I'd rather see that work done in the other repo, rather than here. I think it makes more sense to focus these cooler and more powerful pickers in dedicated locations, instead of trying to maintain support in the core repo. I am trying to push more things out into extensions because there are already too many things in the core repo for me to maintain

[TC72]

@jesseleite I got the highlighting working using code to convert color codes to highlighting but it was a complete mess.
I'd parse it to get the highlights and strip the codes from the text, but then the text could be adjusted based on path shortening and devicons which meant I then had to adjust every highlight again.

The better option would probably be to take a 1st run which strips the codes and you'd be able to make any adjustments. Then the highlights would have the correct location.

But that means parsing the output twice.

What I'd much prefer is if there was something built into telescope which attached highlights to a part of the final string so the highlight code could take care of any adjustments made to shorten paths for example.

JSON would be great in terms of finding the data, but I'm worried it would make a huge difference to the size of data we get back from rg.

[jesseleite]

Sounds good Teej 👍

[tjdevries]

Sounds good Teej +1

<3