Filter hint markers by element text

[lydell]

Original PR description below (which is a bit out-of-date now):

Here is my attempt at fixing #340, heavily inspired by @doy’s #789. A few differences compared to that PR:

• Implements the idea that all hint chars are treated as uppercase in text mode.
• Ignores text mode for the eb command.
• Treats uppercase typed chars as lowercase in non-text mode. This allows text mode users to type uppercase hint chars in eb out of habit.
• When typing a char that gets matched against element text, the new hints for all matching elements are calculated differently than in other cases. The hints are chosen based on a scoring system (from best hint to worst):
  1. Match at the beginning of the text.
  2. Match at the beginning of the text, pass 2 elements.
  3. Match at the beginning of a word.
  4. Match at the beginning of a word, pass 2 elements.
  5. Other substring match.
  6. Other substring match, pass 2 elements.
• Ignores the ignore_keyboard_layout option when typing chars to match against element text.
• Doesn’t treat pressing for example <delete> as typing the characters <delete>.
• No-ops when typing a char that results in zero matches.
• Only ignores keypresses for hints_timeout milliseconds if a hint was activated by matching its text.
• Normalizes whitespace of elements.
• Ignores spaces when matching element text, unless you explicitly type a space (which requires remapping the default <space> shortcut).

@doy, @progandy, @Tunous, @e12e, @erjoalgo, @frangio please test this PR if you can!

TODO:

• More testing.
• Documentation.
• Decide if the hints_text_mode pref should be in VimFx’s settings page in the Add-ons Manager or not.
• Figure out if this is what people want, or if something is missing. EDIT: Here’s a list of potential stuff that has been mentioned below:
  • “Pentadactyl space handling”
  • Hint marker placement improvements (see screenshots below)
  • Image alt text matching (shown in the hint marker)

[doy]

This looks great - I like the improvements! I don't see any obvious bugs after playing around with it for a bit, but I'll keep trying it and let you know if I notice anything.

[doy]

One thing that pentadactyl does (that I've never used personally, but which I noticed when I was fiddling with it to compare) is to not match literal spaces, but split the typed characters on whitespace and require that all of the parts match individually. So for instance, if I type "abc def" in hints mode, it will select all elements which contain both "abc" and "def" as substrings (so "defabc" would be selected, but not "abcfed" or "cbadef"). I'm not sure this is worth replicating (it seems a bit weird and potentially confusing), but just thought I'd mention it as a point of comparison.

[lydell]

Thanks for testing, @doy! I suggest keep testing by using this branch in your main Firefox profile for a week or so. 👍

---

Thanks for mentioning that Pentadactyl difference regarding how it handles typing a space. That’s really helpful. Since you don’t use it, and since it is more complicated to both explain and to implement, I too think that we should leave it out. I also think that behavior sounds confusing.

---

I just added highlighting of the matched text. What do you think of that?

[lydell]

@doy Also, what do you set hints_timeout to? My feeling is that the default is too short for text mode. If so, we might have to split that pref into two.

[doy]

I like the idea of highlighting the matching text, but I don't think the current implementation is very readable with the semi-transparent overlays. Could the hint characters be offset a bit, so that they don't always cover up the first handful of characters? Pentadactyl puts them above and to the left of the link they are attached to, which allows you to see both the hint characters and the link text at the same time. The pentadactyl hint characters are also quite a bit smaller by default, which helps with this.

As for the hint matching input suppression, Pentadactyl uses a hardcoded value of 500ms. I agree that the VimFX default of 200ms (for hints_timeout) feels a bit quick in some cases.

[Tunous]

Great. I'll try to use Firefox with this branch for a few days and share how it performs.

There are my first few thoughts:

---

I agree with @doy that the semi-transparent overlays are a bit annoying. In my opinion they should be offset or made a bit more transparent. Right now both letters in overlays and text under them are almost impossible to read.

Another option could be to only highlight with a border or something which elements are clickable without displaying hint char overlays. They could only appear when users start to hold shift as that's when they are needed. Although it's possible that this behavior could be too confusing for most of users...

---

There is a bug with transparency of hint overlays. After typing first character they turn transparent and require to stop holding shift and hold it again to make them opaque.

---

The behavior of space which @doy described is something what I had in mind when mentioning them. I thinks it's a very useful feature to be able to break mid-word with space and type next one. This mostly helps when clickable elements start with the same text but you don't notice it at first. Simply pressing space and typing one of following words greatly speeds up getting where intended.

Of course it's also possible to use hint chars in that case. I'll have to test it a bit longer to know how often I try to hit space to break words.

---

Default timeout is definitely too short. I type too fast to notice in time when links activate. It already got me few times when I entered ignore mode and got confused why VimFx stopped working. I'm going to find few settings to come up with perfect time for my typing.

[lydell]

I agree that the semi-transparent thing is a bit too hard to read many times. I’ve pushed a new commit now which tries to put hints to the left of text instead. Also, instead of flipping the shift-to-peek-through behavior of holding shift in text mode, I now use ctrl+shift instead.

---

@Tunous Thanks for providing feedback! 👍

Another option could be to only highlight with a border or something which elements are clickable without displaying hint char overlays. They could only appear when users start to hold shift as that's when they are needed.

Another bad thing about the semi-transparent thing that I came to think about is that everyone won’t need to hold shift to be able to type hint chars. I suspect many will use numbers as hint chars. So I think that it’s best to always show all hint markers, and rely on the new positioning instead.

There is a bug with transparency of hint overlays. After typing first character they turn transparent and require to stop holding shift and hold it again to make them opaque.

That depends on how you see it. VimFx has a feature that allows you to hold shift to peek through all hint markers (they get very transparent while shift is held). What I did was simply flipping that behavior when using text mode: The hint markers are quite transparent (but not as much as in non-text mode) by default and become opaque when shift is held. Anyhow, we seem to be leaving this semi-transparent thing anyway, so it doesn’t matter.

I thinks it's a very useful feature to be able to break mid-word with space and type next one. This mostly helps when clickable elements start with the same text but you don't notice it at first. Simply pressing space and typing one of following words greatly speeds up getting where intended.

Would you mind elaborating on this with a real example? Many people like to bring up how they navigate on Hackernews.

Of course it's also possible to use hint chars in that case. I'll have to test it a bit longer to know how often I try to hit space to break words.

I'm going to find few settings to come up with perfect time[out] for my typing.

Please do and report back!

[doy]

The offset hints are an improvement, although I think I still prefer a vertical offset as well. Also, the hints are now in the wrong place when the link wraps - I think they're being placed to the left of the entire bounding box for the link which means that if the link wraps, the hint will show up all the way at the left edge of the paragraph, which is pretty confusing. See
vs
.

Additionally, another thing I just noticed/remembered is that Pentadactyl will also allow using hints to select image links by their alt text (showing the alt text along with the hint characters) - for instance,
. This doesn't come up a whole lot, but it can be useful when it does.

[lydell]

@doy Thanks for the screenshots! That’s super helpful.

I’ve fixed the case of wrapping links now.

I’ll add your remaining items to the first post.

[lydell]

@doy The point of the text mode is to know what to type before seeing the hints, right? That’s not the case for image alt text, so I don’t see how that’s any more useful than simply typing its hint. It just sounds like it might generate lots of really large hint marker that might obscure other things.

[Tunous]

I think I found a prefect timeout which is 400ms. Didn't perform any accidental actions with it.

Some issues:

• Hint overlays hide text in Trello cards. (Demo board)

• If you enter hints mode, press escape and quickly enter hints mode again hint overlays will disappear (but typing text will start selecting). I think it's happening once previous timeout ends.

---

I thinks it's a very useful feature to be able to break mid-word with space and type next one. This mostly helps when clickable elements start with the same text but you don't notice it at first. Simply pressing space and typing one of following words greatly speeds up getting where intended.

Would you mind elaborating on this with a real example? Many people like to bring up how they navigate on Hackernews.

It is mostly useful in websites like google where there are many results with common words. It's often natural to start typing the word which I used when performing search instead of trying to find unique one.

Let's say that my intention is to enter 3rd result from here. In this example I started typing pull as that's what first came to my mind. But I quickly start noticing that it doesn't get me far enough. I would have to first type - and then f. Easier for me is to search for next word, which most likely would be overwrite and then probably code. My final input could be pul ov cod and it is what would make me follow this link.

Alternatively if I started typing overwrite I would have to type that whole word and then the one after it. If I could press space and immediately start typing the following word I wouldn't have to type it all. Instead of overwritey my final input would be like overwy. This is probably much better example than the previous one. That makes being able to skip long words main usage of this feature.

Although, so far I counted only about 3 times where I wanted to break word earlier and I worked around it by using hint chars. Probably it's not really worth trying to implement it considering this alternative ^^

---

Bonus observation: Making typing of special characters like - | ( ) optional (but still possible) would make it much easier to follow some links.

[lydell]

@Tunous Thanks you very much for your detailed feedback! Much appreciated!

Hint overlays hide text in Trello cards.

Fixed.

If you enter hints mode, press escape and quickly enter hints mode again hint overlays will disappear (but typing text will start selecting). I think it's happening once previous timeout ends.

Does this only happen on this branch? If not, please open a separate issue.

[super awesome space example]

I'm totally convinced now. I've implemented something along the lines you describe. Please test and see how it works for you!

Bonus observation: Making typing of special characters like - | ( ) optional (but still possible) would make it much easier to follow some links.

I guess that makes sense, and that it should quite easy to do. The hard thing is to decide the set of characters. Another idea would be to make <tab> or perhaps . work as a wildcard that matches any character. That would avoid the "choosing a set" problem. Thoughts? Do you know how Pentadactyl does it?

I guess this one needs a bit of thinking before rushing off to implementation. :)

---

Regarding the TODO list:

• Space handling – done (might need tweaking though – we'll see)
• Marker placement – IMO it's fine now (I've reduced the default font size with 20% too)
• Image alt text matching – IMO we should leave that one out
• Exposure to users: See below

I've pushed an experiment that I'm really excited about:

• Text mode is on by default.
• The hint chars always take precedence over text matching.
• The default hint chars stay the same.
• This above means that Hints mode works exactly before, until you type non-hint chars.
• Specifying uppercase hint chars is now allowed.
• (If you specify both lowercase and uppercase hint chars, text-transform: uppercase; is not applied to the hint markers.)
• If you prefer matching be text primarily, change the hint chars to uppercase chars only, or numbers only etc.
• The hints_text_mode pref still exists, but it will remain hidden. The idea is that nobody should have the need to toggle it (but I left it there as a safety hatch) .
• There is a separate pref for the timout, set to 400ms per @Tunous suggestion. Vimium uses a hardcoded 200ms timeout. Pentadactyl uses a hardcoded 500ms according to @doy (I couldn't find it in the source code). I tried to find what Vimperator does, but didn't find anything. None of them seem to expose a way to customize that timeout. So I don't think this option should be in the UI.

Many sites (especially forums) have really small pagination numbers. There, it is very difficult to see which hint goes where. Therefore I think it makes sense to have text mode always one. Being able to type for example "5" when I want click the pagination link for page 5 is so useful.

What do you think about this? Does it make sense?

---

Finally, when I was browsing through Pentadactyl's source code, I noticed that it has quite a few hints-related options that might be relevant:

options.add(["followhints", "fh"],
    "Define the conditions under which selected hints are followed",
    "number", 0,
    {
        values: {
            "0": "Follow the first hint as soon as typed text uniquely identifies it. Follow the selected hint on <Return>.",
            "1": "Follow the selected hint on <Return>."
        }
    });

options.add(["hintmatching", "hm"],
    "How hints are filtered",
    "stringlist", "contains",
    {
        values: {
            "contains":       "The typed characters are split on whitespace. The resulting groups must all appear in the hint.",
            "custom":         "Delegate to a custom function: dactyl.plugins.customHintMatcher(hintString)",
            "firstletters":   "Behaves like wordstartswith, but all groups must match a sequence of words.",
            "wordstartswith": "The typed characters are split on whitespace. The resulting groups must all match the beginnings of words, in order.",
            "transliterated": UTF8("When true, special latin characters are translated to their ASCII equivalents (e.g., é ⇒ e)")
        },
        validator: function (values) {
            return Option.validateCompleter.call(this, values) &&
                   1 === values.reduce((acc, v) => acc + (["contains", "custom", "firstletters", "wordstartswith"].indexOf(v) >= 0),
                                       0);
        }
    });

options.add(["wordseparators", "wsp"],
    "Regular expression defining which characters separate words when matching hints",
    "string", '[.,!?:;/"^$%&?()[\\]{}<>#*+|=~ _-]',
{ validator: function (value) { return RegExp(value); } });

• followhints: What about not following hints matched by text if the timeout is set to -1? Then, you could press your first hint char to activate the hint.
• hintmatching: I think we should come up with one nice algorithm and stick with that.
• wordseparators: Do you know how it works? Either way, it's a nice set of characters that the user might not want to type when matching by text.

Thoughts?

[lydell]

@Tunous

If you enter hints mode, press escape and quickly enter hints mode again hint overlays will disappear (but typing text will start selecting). I think it's happening once previous timeout ends.

Does this only happen on this branch? If not, please open a separate issue.

Never mind my question there. It's fixed on master as of commit 0d630e9. Nice catch!

[doy]

Something else that I just realized is that another point in favor of the new space behavior is that it's also the behavior that the awesomebar uses for choosing autocomplete options.

[doy]

I'm not super familiar with the pentadactyl source, but I believe that https://github.com/5digits/dactyl/blob/master/common/content/hints.js#L496 is where the hint completion timeout is set, and then used here https://github.com/5digits/dactyl/blob/master/common/content/hints.js#L536-L542.

[doy]

I did some testing, and it seems like Pentadactyl treats characters like - | ( ) just like any other hint text (like, o|b would match as a substring in a link with text foo|bar, but ob would not). I wonder if the split on space behavior would be sufficient here - it would make o b match.

[doy]

One thing that still makes it a bit obnoxious is that while the hints are placed to the left of text, they are still placed overlapping images, which can make it hard to see what's going on for smaller images:

I'm not sure there's really a lot that can be done here though - small images are going to be hard to avoid covering up regardless. One thing that Pentadactyl does do is use a slightly transparent background for hint characters:
which helps a bit, I think? Will probably require some experimentation to get the right balance there though (to make sure everything is still readable).

[doy]

My guess is that wordseparators is only relevant for the firstletters and wordstartswith settings for hintmatching, and since we only do (the equivalent of) contains, it isn't relevant to us. Highlighting the link that is assigned to the first hint character and making <Enter> select it might be a useful feature, although I'm not sure that I like using a timeout of -1 to force that behavior - feels like it should be its own option (it doesn't really have anything to do with the timeout, it's just behavior that happens to not make a lot of sense to enable when you also have a timeout).

[Kazy]

To extend on using <Enter> to select the first one, Vimperator does it by highlighting the currently selected hint (in purple instead of yellow) when you first hit <Tab>, and has the possibility to switch to the next/previous one using <Tab>/<S-Tab>. I personally got quite used to do it this way instead of typing the hint number.
I also confirms that 200ms for the hint is too quick for me, but 400ms is good.

Thanks for working on this feature !

[lydell]

@doy

Something else that I just realized is that another point in favor of the new space behavior is that it's also the behavior that the awesomebar uses for choosing autocomplete options.

Except that the awesomebar allows matching the words in any order, while my algorithm does not (the words must follow each other). Not sure what’s best here.

[pentadactyl source links]

Thanks!

it seems like Pentadactyl treats characters like - | ( ) just like any other hint text

Thanks for letting us know.

[small images]

While moving the markers out of the way from text was kind of essential to be able to use the text matching feature, that’s not really related to this issue.

Highlighting the link that is assigned to the first hint character and making <Enter> select it might be a useful feature, although I'm not sure that I like using a timeout of -1 to force that behavior - feels like it should be its own option (it doesn't really have anything to do with the timeout, it's just behavior that happens to not make a lot of sense to enable when you also have a timeout).

My thinking was that the timeout doesn’t make sense in that case, so a special value of it could mean something else. Also, I was thinking of not making <enter> follow a hint, but instead to let people type their first hint char.

---

@Kazy

[<enter> and <tab>]

I think a really good example (like the one(s) by @Tunous above) is needed for this to happen. I’m not convinced that this is a good feature.

I also confirms that 200ms for the hint is too quick for me, but 400ms is good.

Thanks! Good defaults are important.

[lydell]

@Tunous Thanks, that makes sense! Again, thank you so much for helping with testing and providing such clear feedback! 👍

[doy]

I just installed the updated version (been busy at work this week), and it seems to work well from the little testing I've done so far - I'll keep you updated. One comment that I have is that using blue for the selected hint makes it hard to see, both because black text on a deep blue background is hard to read and also because the highlight that hints mode uses for matched text is also blue, and so it tends to get lost among the various things that are highlighted: see
for instance.

[lydell]

I now want to merge this into develop rather than to master, so I have to close this PR. Don’t worry it doesn’t mean that the feature got scrapped! You’ll find it in the develop branch from now on.

Feel free to keep commenting here if you’ve got any feedback.

[doy]

@lydell: any thoughts on using a different color than blue? (#803 (comment))

[lydell]

@doy Sorry, I didn't see your comment! Thanks for pinging me about it.

This is the current status on the styling of highlighted markers:

1. I had a really hard time finding something that didn't look terrible. I'm not much of a designer.
2. It is important that it is very easy to distinguish highlighted markers from the rest of the markers, so you more easily see when to stop typing and press <enter> instead.

This is another alternative I considered:

#VimFxMarkersContainer .marker--highlighted {
  filter: hue-rotate(90deg) brightness(90%) !important;
}

I would be more than happy for a contribution of better styling! (Or at least a color suggestion or something.) :)

[lydell]

@doy Here is yet an alternative:

filter: none !important;
--color-primary: black !important;
--color-secondary: white !important;
border-color: var(--color-primary) !important;
box-shadow: 0 0 0 1px var(--color-primary), 0 0 0 2px var(--color-secondary) !important;
color: var(--color-primary) !important;

[lydell]

@Tunous @doy I've pushed several bug fixes and improvements to develop now.

A change is that the entire element text is now selected for highlighted markers (when you have typed at least one non-hint char). That should make it more obvious when you can stop typing and press <enter> instead. Because of that, it is not that important to highlight the markers themselves in a very distinct way anymore. So now they get a diagonal white stripe.

[doy]

I find this also pretty hard to see - the difference between a thing highlighted because it's part of the thing that you're typing and a thing highlighted because it's the first choice is not super apparent. I like the first option in the previous comment better, I think - differences in color is easier to see than differences in the amount of thing highlighted. Maybe keeping the "select the whole thing" behavior that is currently implemented, but change the highlight color? (Is that a possible thing?)

Also, the current styling for the selected hint marker looks pretty weird on my machine, not sure if this is intentional?

[lydell]

@doy Could you please help out and come with a concrete suggestion. This styling problem is driving me crazy here :)

[doy]

Of the options given so far, I think #803 (comment) is the best (and I would be happy with that as the implementation).

For a concrete suggestion that I think would probably be better, I think it'd be nice to have something more along the lines of what Pentadactyl does (although with some modifications). Pentadactyl highlights the entire element for all matching elements in yellow, and highlights the entire selected element in green:
This is very easy to see, but loses the visibility that the current implementation has around highlighting the part that actually matches, which I really like. My suggestion would be to highlight the entire elements like Pentadactyl does, but overlay the existing match highlight that currently exists, so that we can easily see which elements are matched and also which part of the elements triggered the match.

[lydell]

Of the options given so far, I think #803 (comment) is the best (and I would be happy with that as the implementation).

Thanks. We’ll probably go with that, then.

My suggestion would be to highlight the entire elements like Pentadactyl does, but overlay the existing match highlight that currently exists, so that we can easily see which elements are matched and also which part of the elements triggered the match.

1. I don’t know of a good way to highlight in multiple colors.
2. Why does it matter to see which parts of the element triggered the match?

[lydell]

@doy The latest iteration:

[doy]

Seeing which part of the element triggered the match makes it a lot easier to see what other part of the text isn't matched in case I need to keep typing to get a more specific match. It's not a huge deal though - I'm happy with the latest iteration.

[Tunous]

Seeing which part of the element triggered the match makes it a lot easier to see what other part of the text isn't matched in case I need to keep typing to get a more specific match.

I agree with this. Didn't notice that the latest iteration had whole text highlighted. For me the hint colored in green is good enough.

[lydell]

So, what happened to me was that the hint marker was too far away from the part of a link I was trying to match, so I didn’t notice it becoming green. That’s why I came up with the idea of highlighting the entire element (as a complement).

Are you saying that when your initial guess on which part of a link to type didn’t work out, you start looking at other links to figure out what to continue typing, and are then bothered by one (or a few) of them being fully highlighted?

Also, which is more true to you:

1. The highlight-entire-element feature is nice, and only occasionally bothers me, so it’s fine.
2. The highlight-entire-element feature is nice, but too much in the way, so I’d rather see it removed.
3. The highlight-entire-element feature is useless – please get rid of it!

[Tunous]

Option 1. It just makes it slightly easier to see what to type in some cases. But I'm sure highlighting the entire element will come more useful than that.