Integrate fzf-completion with fzf-tab-complete

[maximbaz]

I have configured fzf completion to be smart when suggesting files and directories, it includes hidden folders, skips ignored files, and many more:

FZF_COMPLETION_TRIGGER=''                                # ctrl-t goes to fzf whenever possible

# Default file search commands
export FZF_DEFAULT_COMMAND='fd --hidden --follow --type=f'
_fzf_compgen_path() { fd --hidden --follow --type=f }

# Default directory search commands
export FZF_ALT_C_COMMAND='fd --hidden --follow --type=d'
_fzf_compgen_dir() { fd --hidden --follow --type=d }

. /usr/share/fzf/completion.zsh                          # load fzf-completion

With the above, vim <tab> shows fzf with all the files I want to see, and cd <tab> shows fzf with all the directories I want to see, including nested files and directories.

The rest of the completion is rather broken, e.g. man <tab> or pass <tab> also show fzf with the files starting from the current directory, just like with vim <tab>.

I would like to make fzf-tab kick in when this happens, i.e. for file and directory completions keep using fzf-completion because it knows well how to select files and directories, but for all the rest of the completions I want fzf-tab-complete to be executed.

Is it possible to achieve this?

Alternatively, is it possible to teach fzf-tab which files and directories it will present me? I would be fine to stop using /usr/share/fzf/completion.zsh if fzf-tab can also be told to use fd --hidden --follow --type=f for example.

Let me know if this makes sense, and thanks! :)

[romkatv]

I wanted to ask the same question since the day I discovered fzf-tab (an awesome project by the way!) I'll try to paraphrase what you are asking.

Is it possible to hook _files in fzf-tab and use fzf there directly?

For example, right now, when I type type grep test and hit TAB (bound to fzf-tab-complete), completion candidates are generated by _files and then fzf-tab presents them with fzf. In the hypothetical world where _files is hooked by fzf-tab, _files would still be called but it would show fzf right there before _files returns. The file selected in fzf by the user would be the only completion candidate inserted by _files.

This would have the obvious advantage over fzf-completion of understanding zsh and command syntax. It would also have the advantage over the current version of fzf-tab of allowing you to pick the right file in one go rather than completing one path segment at a time.

My knowledge of Zsh completion system is too rudimentary to know whether this is even remotely possible.

[Aloxaf]

I think this is a question about compsys, not fzf-tab.

A simple solution is override the default completion functions/completer:

_files() {
    local files=($(fd --hidden --follow --type=f))
    compadd -a -f files
}

_cd() {
    local dirs=($(fd --hidden --follow --type=d))
    compadd -a -f dirs
}

Or if you just want to enable it for some specific commands:

# save the original _files completer
autoload +X -Uz  _files
functions[_files_orig]=$functions[_files]

_files() {
    if zstyle -t ":completion:$curcontext" use-fd; then
        local files=($(fd --hidden --follow --type=f))
        compadd -a -f files
    else
        _files_orig
    fi
}

zstyle ':completion:*:vim:*' use-fd true

Hope this can help you.

[romkatv]

Thanks for the working code snippets! This is super helpful.

_files() {
    local files=($(fd --hidden --follow --type=f))
    compadd -a -f files
}

This won't take advantage of streaming mode in fzf, so it can result in noticeable freezes when hitting TAB. Breadth-first traversal of files piped into fzf has great UX. Is there a way to use it here?

[Aloxaf]

Umm, compsys itself doesn't support streaming mode, and nor does fzf-tab.

To take advantage of streaming mode of fzf we should call it in advance.

_files() {
    compadd -f -- "$(fd --hidden --follow --type=f | fzf)"
}

On the contrary, this can't take advantage of compsys&fzf-tab (eg. list-colors).

[romkatv]

To take advantage of streaming mode of fzf we should call it in advance.

Yes, that's what I've been trying to suggest.

On the contrary, this can't take advantage of compsys&fzf-tab (eg. list-colors).

Why not? If this is done from within fzf-tab, then it should be possible, no?

[maximbaz]

To take advantage of streaming mode of fzf we should call it in advance.

Thanks, this is close to be very good already. This works with vim <tab>, but if I do vim prefix<tab> then completion will be shown 3 times for me (fzf opens, I press Enter, fzf closes and opens again, and this repeats 3 times in total). I think I can figure out on my own how to feed prefix to fd, but can you give me a hint why fzf shows up 3 times?

[romkatv]

I don't think it's close to being solved. The problem is that _files can be called many times by a completion function.

[romkatv]

I'm not trying to offer suggestions how this can be solved. I only know enough of the completion system to know that there is no easy solution. My goal is to interest @Aloxaf enough in this problem so that he solves it 😉 It would be fantastic if TAB behaved the same as it does now in fzf-tab except that file completions had all possible candidates preexpanded and streamed into fzf.

[Aloxaf]

If this is done from within fzf-tab, then it should be possible

Yes, that sounds great.
But currently, fzf-tab's function is built for compsys and are all blocking. Considering that compsys is blocking, to support streaming mode I think we need to bypass compsys to add candidates. Then we finally go back to something like #41 (support another completion backend).

[maximbaz]

For context, why did you revert #41? 🙂

[romkatv]

Would it be possible to hook compadd and stream to fzf from there?

[romkatv]

For context, why did you revert #41?

See discussion in #48.

[Aloxaf]

but if I do vim prefix then completion will be shown 3 times for me

I can't reproduce this, but maybe you could add a flag to prevent _files from called multiple times? (umm, that may also be a little complex)

And I come up with another rough approach, define a completer so that it will only be called one time:

FZF_FILES=(vim exa)

_fzf_files() {
    if (( $FZF_FILES[(I)$words[1]] )); then
        compadd -f -- "$(fd --hidden --follow --type=f | fzf)"
        return 0
    else
        return 1
    fi
}

[Aloxaf]

Would it be possible to hook compadd and stream to fzf from there?

That's what's fzf-tab-completion does. But normally compadd will receive a group of candidates at a time. If we want it to stream every single candidate to fzf, then we have to pass the candidate to compadd one by one, which will be slower.

Like this:

_files() {
    fd --hidden --follow --type=f | {
        while read line; do
            compadd -f -- $line
        done
    }
}

[maximbaz]

I figured out what caused fzf to appear multiple times, I had the following configs back from when I used prezto (seems like I can remove them now that I use fzf-tab):

zstyle ':completion:*' completer _complete _match _approximate
zstyle ':completion:*' matcher-list 'm:{a-zA-Z}={A-Za-z}' 'l:|=* r:|=*'

[romkatv]

That's what's fzf-tab-completion does.

Why isn't fzf-tab doing it? What's the downside?

But normally compadd will receive a group of candidates at a time. If we want it to stream every single candidate to fzf, then we have to pass the candidate to compadd one by one, which will be slower.

If fzf-tab could display completion candidates pushed with compadd in a streaming fashion, we could change _files (and other completers) if we wanted to take advantage of these streaming capabilities. We don't have to push everything at once or everything one-by-one. There is a middle ground that could allow us to reap the benefits of streaming mode and avoid the overhead of many compadd calls at the same time. For example, buffer the output of fd and pass a batch to compadd every 100ms.

The real question is whether fzf-tab can push completions from compadd directly to fzf in streaming mode. What are we going to lose if fzf-tab learns to do this?

[Aloxaf]

What's the downside?

I didn't choose this way mainly for speed. Another is for some function like getting the common prefix of candidates or sort the candidates or group color. They need the whole candidates to calculate.

Ummm, let me think, if fzf-tab need to support streaming mode.

1. Though it may lose some speed, the user experience will be better especially when there is a large number of candidates.
2. I think I can modify the _fzf_tab_get_candidates to only calculate the prefix among the first N candidates. That should be enough at most time.
3. It can only sort the single group, not the whole candidates.
4. There can still be colors, but there won't be group headers.

Looks still attractive, especially when I found zsh is still lack of speed when trying to support list-colors.

It might need another version of fzf-tab (fzf-tab-streaming.zsh?) to let users switch from them?
Anyhow, I will give a try.

[romkatv]

💯 🚀 🏆

Give a shout if you need someone to test stuff or bounce ideas off.

only calculate the prefix among the first N candidates

How would this work if the first N candidates share a long prefix but the following candidates have a different first symbol?

There can still be colors, but there won't be group headers.

I think this is fine to begin with. If your ideas w.r.t. streaming fzf-tab pan out, it might be worth it to add group support to fzf. Basically, allow headers in the middle and not only at the top. This could be useful to other users of fzf, not just to fzf-tab.

[kevinhwang91]

I need the stream mode. If the input source is large, fzf-tab will hang.

Such as touch a{1..100000}, the response of fzf-tab is more terrible than exa | fzf.

[kevinhwang91]

What I crave most is git completion(such as git log), but the current response of fzf-tab makes me a bit uncomfortable.

[jakubfijalkowski]

@kevinhwang91 I hacked some code that streams the fd results to fzf for files and directories, i.e. it circumvents fzf-tab & compadd for _files and _cd completers. This is a variation on what @maximbaz and @Aloxaf have shown above but with multitude of edge case handled. Doesn't really solve the problem but makes it faster for some of the cases.

[romkatv]

@maximbaz et al

I've implemented something along the lines of what's been discussed here. The implementation is very hacky but seems to work well. The implementation is currently coupled to zsh4humans/v3, which is very much bleeding edge and not recommended for serious use. Nonetheless, I'd be grateful if you tried it in a docker container and let me know if this works for you and what you would like changed. Some of the hacks could be rolled into fzf-tab. Others might become their own plugin.

To try it in docker:

docker run -v $HOME:/host -e TERM -e COLORTERM -w /host -it --rm debian /bin/bash -uec '
  apt-get update
  apt-get install -y curl zsh
  [[ -e /host/.p10k.zsh ]] && cp /host/.p10k.zsh ~/
  exec sh -c "$(curl -fsSL https://raw.githubusercontent.com/romkatv/zsh4humans/v3/install)"'

Within the container, /host is your home directory on the host. Don't delete things in there as they will disappear on your host. You can do whatever in ~ though (a.k.a. $HOME). Try doing things like cd /host/<TAB>, or ls /os<TAB>.

Tab works just like in fzf-tab. When it opens fzf, the first entries are the same as in stock fzf-tab.

• cd /host/<TAB> gives all directories under / (not yet recursive).
• ls /os<TAB> gives "host" as the only completion because it's the only file under / that has "os" as substring.

When there are directories among entries, their content gets added at the bottom, recursively. This is the difference with the stock fzf-tab. Additional completion candidates get streamed into fzf but the top candidates are just like before.

You can accept suggestions with Enter or Tab. The latter will immediately trigger another completion if the inserted completion is a directory. This is similar to "continuous trigger" in fzf-tab but works slightly differently. Binding to Tab is very natural. This binding is helpful when you do something like ls /<TAB> and it starts traversing the whole disk. Since the top-level directories are always at the top, you can hit Tab on /usr without waiting for the whole /dev to be traversed (assuming that you wanted to ls something like /usr/local/lib). This will effectively narrow search to /usr as a sort of shortcut where you'll be able to find local/lib quickly.

You can also accept file completions with Alt+Enter. This will simply insert the query from fzf as completion. This is nice when hitting Tab gives you lots of entries that all start with very-long-prefix- and you actually want very-long-prefix-* on the command line. With the stock fzf-tab you either have to close fzf and lose very-long-prefix- or accept very-long-prefix-very-long-suffix and then manually delete very-long-suffix. In zsh4humans you can just press Alt+Enter.

Recursive file completions are only smart enough to figure out whether you want just directories or everything. They currently don't support file patterns and ignore lists. (This can be added.)

File completions are colorized, like in fzf-tab. I wrote a very fast colorized that processes 70-100k files per second on my machine. It also fixes a few bugs that existed in fzf-tab colorizer. (This part should be relatively easy to backport to fzf-tab.)

You can also notice that there is no prompt flickering when completing, unlike in the stock fzf-tab. And also that autosuggestions and syntax highlighting are always correct and faster than normal. zsh4humans is a giant pile of hacks that does awful violence to plugins in order to integrate everything nicely.

If things are confusing, look at ~/.zshrc. There are a few options that you can easily tweak to reduce the amount of confusion.

If you want to try it without docker, backup your zsh startup files and run this:

exec sh -c "$(curl -fsSL https://raw.githubusercontent.com/romkatv/zsh4humans/v3/install)"'

[romkatv]

At some point I also had the following implementation:

• Tab works like in the stock fzf-tab.
• When fzf appears, you can press Ctrl+R (for Recursive) and extra entries will start streaming at the bottom. This happens in a nice way without flickering.

I tried it because I thought that automatically streaming recursive directory content would be bad when what you are looking for is just a single file completion. After playing around with automatic streaming I realized these reservations are unfounded. Things that really help with it:

• top-level completions are always immediately available
• you can accept suggestions before file traversal finishes
• you can accept fzf query as completion
• file traversal and colorization is very fast

[maximbaz]

Thanks @romkatv! In a very short summary, I love it! I tried everything you mentioned and it feels great. The only annoying thing is something you have mentioned, a non-recursive suggestions on the first Tab: in home dir press cd <Tab> and it would only suggest one level of directories, but continue typing .local and press Tab again, and now the list of suggestions has a full tree of everything under ~/.local with all levels deep. This, and what we discussed some time ago about extending <Alt-Down> with frecency list of previously visited directories, is all I can think of so far!

[romkatv]

@maximbaz Thanks for giving it a try and posting feedback.

The only annoying thing is something you have mentioned, a non-recursive suggestions on the first Tab: in home dir press cd <Tab> and it would only suggest one level of directories, but continue typing .local and press Tab again,

I think I've confused you. When you press Tab, directories are traversed recursively right away. However, there is special logic w.r.t. directories starting with a dot.

1. If you don't have dot_glob option set, these directories are ignored.
2. If you do have dot_glob set, hidden directories are shown but not traversed.

Here's an example. Suppose you have this directory structure in your home directory:

.
├── .a
│   └── b
└── c
    ├── d
    │   └── g
    └── .e
        └── g

If you type cd <TAB>, you'll see these directories:

.a/
c/
c/d/
c/d/g/
c/.e/

You won't see these directories because they are nested inside hidden directories:

.a/b/
c/.e/g/

I have dot_glob set in my config and originally I had a simpler logic -- all directories were traversed. This was quite annoying because apparently I have over half of all files in hidden directories I don't care about. So I'd changed it to what I've described above. It's easy to add a option for this though.

This, and what we discussed some time ago about extending <Alt-Down> with frecency list of previously visited directories, is all I can think of so far!

Yeah, I remember that. I haven't found good UI for this yet but I have more tools now that I could use once I figure out how this feature should work. Intuitively, injecting frequently used files and directories isn't much different from injecting files and directories by recursing into suggested directories (this is what I've done here). The problem is that doing this blindly wrecks prefix matching and ranking. I could do it with an extra binding. E.g., instead of Tab you'd press Shift+Tab and it would offer files / directories (depending on the command) that you've used in the past. Here "used" can be quite general. Running any command while having /foo/bar as current directory is "using" /foo/bar. Running vim x/y is "using" /foo/bar/x/y, etc. Then you could cd to a recent directory with cd <Shift+Tab>. I don't know what should be the alternative binding for <kbd>Alt+Down</kbd> or if there is a way keep just one binding.

[maximbaz]

I have dot_glob set in my config and originally I had a simpler logic -- all directories were traversed. This was quite annoying because apparently I have over half of all files in hidden directories I don't care about.

Oh I see! Might I suggest an alternative and more precise approach, where instead of treating all hidden folders specially, just respect .gitignore and .ignore files in the current directory and in $HOME? I know this will involve a bit more work with parsing and filtering, but on the other hand tools like fd and rg already support these files, and so by respecting those ignore files the behavior of fd, rg --files and <Tab> / <Alt-Down> will all be consistent. This way e.g. if you don't want to see anything under ~/.cache, you would put .cache in ~/.ignore.

Regarding frequently used folders, I agree it's a complicated matter, I also don't have a clear idea in my mind... I do like that you are building more tools that will be useful for this feature 🙂 But yeah maybe lets leave this out for a moment.

I have another idea or request, more relevant to the completions scope. Would it be possible to not show fzf when there is a single match? Imagine I'm typing vim p10k<Tab> and there is only one possibility, p10k.zsh, I don't need a fuzzy filter for that, I just want the whole file to be completed. Same if it's a folder, if I type cd .conf<Tab>, if only .config folder matches, let the first <Tab> just complete it up to .config without opening fzf, if I want to open fzf with the list of nested folders I will just press <Tab> a second time.

I guess all I described might be a special case of #8, which I also agree with and which would be nice to have. That ticket explains everything, if there is a common prefix let the first <Tab> just complete the common prefix, and let the second <Tab> open fuzzy filter.

[romkatv]

Oh I see! Might I suggest an alternative and more precise approach, where instead of treating all hidden folders specially, just respect .gitignore and .ignore files in the current directory and in $HOME?

.gitignore defines patterns for things that must not be committed to a Git repository. What we need is a pattern for things that are unlikely to be invoked as commands or passed as arguments to commands. These aren't the same. ~/foo/.gitignore may contain bin/, and yet after running make in ~/foo I'm quite likely to execute bin/x86-64/opt/blah. It makes sense to honor .gitignore by default in rg because you almost always want to search in source code rather than in generated artifacts. However, using artifacts, or analyzing them (ls, objdump, etc.) is very common.

I can add customization points that would allow you to specify that such and such directory should be ignored, while some other directory should always be recursively descended. For now, though, I'm trying to figure out what the default behavior should be so that it requires as little customization as possible.

I have another idea or request, more relevant to the completions scope. Would it be possible to not show fzf when there is a single match? Imagine I'm typing vim p10k<Tab> and there is only one possibility, p10k.zsh, I don't need a fuzzy filter for that, I just want the whole file to be completed.

Oh, that's a bug. Fixed.

Same if it's a folder, if I type cd .conf<Tab>, if only .config folder matches, let the first <Tab> just complete it up to .config without opening fzf, if I want to open fzf with the list of nested folders I will just press <Tab> a second time.

This is a trade off. To decide whether to just complete .config/ or expand it recursively we need to estimate how often you would want to stop on a directory vs completing all the way to a leaf. I think in this case the chances of stopping on directory are pretty high, so I've implemented what you suggested.

I guess all I described might be a special case of #8, which I also agree with and which would be nice to have.

This is easy to do but here I'm not sure it'll be a win. If the first tab completes to a common file prefix, what are the chances that you will not press another tab immediately. I think they are very low. It does sometimes happen though, and for that there is alt+enter in fzf. If you just want to insert prefix, going through tab -> fzf pops up -> alt+enter is surely more work than plain tab, but it happens rarely. On the other hand, pressing tab twice where one would do isn't as annoying but would be very common.

I'm undecided here, so no changing the code yet. There are also some middle ground solutions such as inserting the prefix if it's longer than N.

[maximbaz]

Agree with .gitignore, but ~/.ignore might still worth considering. But in any case, as long as there is an option to say "I want all files and folders except these", I will be happy 🙂

I agree with everything else you wrote, if I can think of a compelling argument in favor of completing up to a common prefix I will let you know.

I guess now I will be eagerly awaiting when you declare your work stable for day-to-day usage 🙂 If you will make it into plugins that would be simpler to choose from (instead of fully switching to zsh4humans) it would be nice, although I might just as well consider switching to zsh4humans. Thanks once again for everything you do 👍

[romkatv]

Agree with .gitignore, but ~/.ignore might still worth considering.

I misread .ignore as .gitignore in your previous comment.

But in any case, as long as there is an option to say "I want all files and folders except these", I will be happy

I was thinking of zstyle option rather than separate .ignore files.

The standard completion system respects ignored-patterns. Currently recursive directory listing doesn't respect it but it's on my TODO list to fix this. Once that's done I can add not-recursed-patterns or something along those lines.

I guess now I will be eagerly awaiting when you declare your work stable for day-to-day usage

When zsh4hmans/v3 is ready, I'll open an issue for feedback and ping you on it. Your feedback was instrumental to several crucial p10k features and I'm hoping we can achieve similar synergy with z4h.

If you will make it into plugins that would be simpler to choose from (instead of fully switching to zsh4humans) it would be nice

I don't know yet how it'll shape out but there are a few things I do know. zsh4humans will stay either way and it will keep providing "deep integration" (a.k.a. dirty hacks) for various plugins it uses. The value from this is just too high to pass on. Naive combination of zsh-syntax-highlighting + zsh-autosuggestions + fzf-tab + powerlevel10k results in many inefficiencies, minor bugs and rough edges. These are very difficult to overcome by changing individual projects because each project is meant to work in so many different environments and configurations. But within zsh4humans I can make those changes because I control the whole environment. E.g., I can remove zle -R from fzf-tab and powerlevel10k because I can guarantee these widgets are never wrapped. This is just one example of many.

I'm almost certain I'll send a few PRs to fzf-tab where improvements have no trade offs and make sense for all fzf-tab users.

There is a 50/50 chance zsh4humans will keep using fzf-tab vs having an embedded stripped-down and modified version of it. There is a very small chance this version will be available as a separate plugin.

although I might just as well consider switching to zsh4humans. Thanks once again for everything you do

I'll try to win you over. v3 is the first version I'm using myself and I honestly think it's very good. I've hand-optimized and polished all basics and implemented several advanced features that bring shell experience on the whole new level. For example, you can set up ssh to automatically teleport your environment to the remote host and pulls your command history back to the local host.

P.S.

If you'd like to follow up w.r.t. zsh4humans, please do it via zsh4humans repo. Let's keep this thread focused on fzf-tab.

[ahmadie]

Using fzf with
cd **(tab)
Would trigger all possible paths in streaming manner.

[ryuheechul]

Using fzf with
cd **(tab)
Would trigger all possible paths in streaming manner.

What an awesome solution! 👏🏼

[fjchen7]

Customized FZF fuzzy completion is a workaround to modify completion candidate list while working well with fzf-tab.

FZF_COMPLETION_TRIGGER=''

_fzf_complete_git() {
    ARGS="$@"
    local branches
    branches=$(git branch -vv --all)
    if [[ $ARGS == 'git co'* ]]; then
        _fzf_complete --reverse --multi -- "$@" < <(
            echo $branches
        )
    else
        eval "zle ${fzf_default_completion:-expand-or-complete}"
    fi
}
_fzf_complete_git_post() {
    awk '{print $1}'
}

Now zsh can show the candidates from the command we configured when typing git co <tab>.

[intelfx]

I just want to throw my own workaround into the pile.

What I did is a cute hack to invoke the default fzf-completion widget with a double <Tab> instead of a trigger sequence:

fzf-completion-notrigger() {
	# disable trigger just this once
	local FZF_COMPLETION_TRIGGER=""
	# if fzf-completion can't come up with something, call fzf-tab-complete
	# instead of the default completion widget (expand-or-complete).
	#
	# FIXME: triggers an infinite recursion on an empty prompt
	# _zsh_autosuggest_highlight_reset:3: maximum nested function level reached; increase FUNCNEST?
	#
	#local fzf_default_completion='fzf-tab-complete'
	fzf-completion "$@"
}
zle -N fzf-completion-notrigger

# Set an aggressive $KEYTIMEOUT to make usage of single <Tab> less miserable
KEYTIMEOUT=20
# Bind double <Tab>
bindkey '\t\t' fzf-completion-notrigger
# Bind Ctrl-Space in case I am unable to use double <Tab> due to a combination
# of the aggressive $KEYTIMEOUT on a slow link.
bindkey '^ ' fzf-completion-notrigger

Note that if fzf-completion fails to come up with something, it will call the default zsh completion widget. I tried to work around this as well, but for some reason it ends up in an infinite recursion. For myself, I fixed this by reordering fzf-tab to load before fzf-completion (contrary to fzf-tab README). See intelfx/dotfiles@425014b and intelfx/dotfiles@25aa473.

---

Anyway, this limitation makes sense — fzf-completion does not use compsys to generate the response, that's why it is able to invoke fzf in a streaming manner, but that's also why it will never be able to integrate with compsys (and, by extension, with fzf-tab). When fzf-completion runs, there is no compsys context, therefore it is fundamentally unable to determine whether it needs to do its thing or defer to fzf-tab. That's why it needs a human-based trigger.