Switch to split markdown parser

[MDeiml]

My new split markdown parser works quite well now (better than the non-split one). This PR changes the config to point at the new branch because I want to keep the old branch for now as other projects are using it.

It also adds some other changes that are needed for the two parsers to work:

1. An exclude_children! directive, that might also be very useful for something like Injected markdown incorrectly highlights indented docstrings #2212
2. Splitting markdown queries into block and inline ones and add the injection for inline into block grammar
3. Add an include_dir option to parser configs (needed, because the two grammars don't live in the repos root directory)

[clason]

Do you want to add yourself as a maintainer so people know whom to ping when things go wrong?

[MDeiml]

Sure good idea

[clason]

Also, please don't forget to mark this as a breaking change (feat(markdown)!: switch to split parser) as this will invalidate downstream queries.

[MDeiml]

I also noticed a bug (which probably should be solved separately from this PR): A node from an injected language used for highlighting can have a range that might include bytes that lie outside of the included range. E.g. with this

> *Foo
> Bar*

The emphasis spans over the second >, but that one is not part of the included range for the inline content. It is still highlighted as emphasis.

We would need to calculate the intersection between included range and node range for every node used in an injected language and use that instead.

[theHamsta]

@MDeiml wow, awesome that we now have an exclude_children! directive. We can of course merge it. But I'm wondering whether we should just make it always the default as this is the way upstream tree-sitter behaves (except "injection.combined" is set https://tree-sitter.github.io/tree-sitter/syntax-highlighting#language-injection).

[MDeiml]

I also think excluding children is the better default, but making this the default would probably need changes (and some discussion) in neovim itself. Maybe it's better to first test this directive here and later make a PR to neovim?

[kyazdani42]

looks good to me otherwise :)

[theHamsta]

I'm having similar performance issues as with the old parser. Was the intention of the split to improve performance?

Wrong alarm. I accidentally used the old markdown parser (was left installed as an artifact of testing a different PR at alternative install location). It seems to be slow still with markdown_inline but that might also be our injection implementation that does no incremental parsing.

[MDeiml]

It's a bit faster, but still slow. I think parsing time after edits is still linear with file size (should be linear with size of edit), because all inline ranges with more complicated elements like emphasis or code spans still get reparsed. This could be fixed, by detecting which inline ranges actually got edited and then just reusing the old trees for all the other ones. But that is something that tree-sitter itself should be able to detect, so maybe I'm gonna create a PR in for that in tree-sitter.

[clason]

Just on the off-chance: will tree-sitter/tree-sitter#1783 be of any help?

[MDeiml]

I don't think so. There is no "syntactically wrong markdown", so error recovery is not relevant.