Slow Markdown Highlighting

[pascalkuthe]

Summary

Since the recent update to the markdown grammer by @David-Else, I have noticed extreme slowdown on markdown files. For example if I hold down backspace and then let go, helix keeps deleting for roughly 0.5seconds.
I have a very beefy PC so this should really not be happening and do not see issues for any other language.
To isolate problems I have disabled the LSP but the problem persists until I turn of syntax highlighting (set-language text) completly.

Looking at the flamegraphs (with LSP disabled) I noticed that a ton of time seems to be spend on parsing TS compared to other languages (rust in this example)

opening and deleting some text in a md file:

opening and deleting some text in a rust file:

Reproduction Steps

Open any markdown file with syntax highlighting enabled and started deleting text

Helix log

No response

Platform

Linux

Terminal Emulator

alacritty

Helix Version

22.08.1-165-g4d4be0e8

[David-Else]

With helix 22.08.1 (4d4be0e8) I am not noticing:

For example if I hold down backspace and then let go, helix keeps deleting for roughly 0.5seconds.

I am led to believe that Markdown is a bit slow for now and the author intends to improve it in the future. Did you try the flamegraph on the version before the Markdown update?

I am also on Linux (AlmaLinux 9) using the latest Kitty terminal.

[pascalkuthe]

Ok I tested again before the update to the markdown grammar.
I must have misremembered because the same slowdown still appears at that commit.
I guess the markdown grammar is just slow :/

I am very surprised that you do not notice this.
My PC is very beefy (ryzen 9 3950X + GTX 1080 + 64GB ram, fedora 36) so I would expect this to be worse for other people.
I tried using kitty instead of alacritty but that doesn't change anything.

Its very noticeable when comparing between editing rust and working on a somewhat large markdown file (1k loc). I sit at a constant 10% CPU utilization (which is a lot considering my CPU). The second I close the file (or switch to a rust file for example) the utilization drops to1%.

Just an example which might better highlight my problem: When I edit the following line from the helix changelog:

- The `extend_line` command has been replaced with `extend_line_below` and a new `extend_line` command now exists ([#3046](https://github.com/helix-editor/helix/pull/3046))

I start holding backspace at the EOL and let got at the first ( so the line should look like this:

- The `extend_line` command has been replaced with `extend_line_below` and a new `extend_line` command now exists ([#3046](

but instead helix keeps deleting:

- The `extend_line` command has been replaced with `extend_line_below` and a new `extend_line`

This seems pretty extreme to me in terms of lag.
Perhaps something else is going on? I completely re-downloaded/recompiled all TS grammars and still get the same problem.

[David-Else]

I tried your test with the changelog and my results were even worse, it deleted the entire line! Perhaps @MDeiml could comment? Some threads maxed out:

I also tried on some technical writing I was working on which is mostly just prose of 700 lines with many embedded code blocks. I am using the Gnome system monitor and it reads 0.04-0.13% for Kitty running Helix with the file being edited/marksman language server running on an ancient Intel Haswell i7. I found no lag editing that.

Could it be all the links in the changelog are causing the slowdown? There is something going on...

[pascalkuthe]

Okay its definitely the links, by simply removing all of them completely fixes the lag.
Now we atleast know whats causing it :D

[David-Else]

I tried the same CHANGELOG.md test with Neovim 0.7.1 and the results were much better, just a slight lag. The thread use was also different:

I think it is using the same split parser. It is certainly using treesitter with MDeiml/tree-sitter-markdown.

:TSInstallInfo

markdown          [✓] installed  

[pascalkuthe]

Are you sure that this is really the same grammar?
I removed all links from the CHANGELOG.md file and compared the flamegraphs:

with links/lag

without links/lag

To me this looks like helix simply spends a ton of time calling the TS parser during the deletions.
I can't really see how nvim would improve upon this, if the TS grammar is the same.

[David-Else]

Are you sure that this is really the same grammar?

Yeah, I checked the grammar commit nvim-treesitter/nvim-treesitter#3048 against the date I last updated the plugin and it is the new split version:

Date split parser was added to nvim-treesitter:

Modified date on my plugin folder

The fact it is working so much better in Nvim makes me think there is something wrong with the Helix implementation.

I don't want to update my plugins again in case my entire nvim explodes due to the recent release of 0.8... one reason I love Helix :)

** EDIT maybe there is a way for me to print the exact version in Neovim, but I can't find it :(
** If you want to try Neovim the appimage works very well on Linux, but don't forget to install and setup treesitter as it is not default

[pascalkuthe]

Thanks for looking into this :) I know my way around the helix highlighting implementation so I will try to find out what helix is doing differently here and fix the bug. I am going to have to look at nvim-treesitter to find out what they are doing differently... Which is going to be a pain (lua + C is not nearly as nice to read as rust)

[pascalkuthe]

I think I found the cause:

The links are highlighted using injections and injections are currently not parsed incrementally.
The original PR (#3108) by @MDeiml that added the split markdown parser actually mentions precisely this improvement:

It's related to this TODO:

helix/helix-core/src/syntax.rs

Line 724 in 681c0a9

// TODO: we should be able to avoid editing & parsing layers with ranges earlier in the document before the edit

Just going one step further and also avoid parsing layers that are after the edits

I am looking into addressing this but its not trivial.

[pascalkuthe]

This is fixed by #4146.
While it looked a bit daunting at first, it actually was a pretty easy fix.
I am a bit surprised you are not seeing the same problems with nvim.
From nvim-treesitter/nvim-treesitter#2916 and neovim/neovim#14762 it does sound as tough the same issue appears there.
I setup nvim locally and while the problem wasn't quite as bad as with helix the lag was still noticeable.
However with helix and #4146 I get no lag at all (so much better then nvim)

[MDeiml]

I'm sorry I didn't participate in the discussion. Just wanted to say thank you, I think this was the best solution. The markdown grammar is slow, and that probably can't be fixed, but making parsing fully incremental should make any tree sitter parser sufficiently fast. I'm guessing this also sped up a lot of other grammars 👍