Heredoc + line escape causes problems

[alexr00]

Does this issue occur when all extensions are disabled?: Yes

Metadata

Version: 1.85.1
Commit: 0ee08df0cf4527e40edc9aa28f4b5bd38bbff2b2
Date: 2023-12-13T09:47:11.635Z
Electron: 25.9.7
ElectronBuildId: 25551756
Chromium: 114.0.5735.289
Node.js: 18.15.0
V8: 11.4.183.29-electron.0
OS: Linux x64 5.15.0-91-generic snap

Bug

Initial variant

Code:

#!/bin/bash

cat << \
────────────────────────────────────────────────────────────────────────────────
oh no's
/* this comment and everything
 * after has wrong colors
 */
────────────────────────────────────────────────────────────────────────────────

function foo() {
    echo "Function foo"
}

foo

As you can see, even here in Github everything after the oh no's has wrong colors.

Other variant

Code:

#!/bin/bash

cat << \
EOL
oh no's
/* this comment and everything
 * after has wrong colors
 */
EOL

function foo() {
    echo "Function foo"
}

foo

Working example

Without single quote

This works in VSCode but not on Github.

Code:

#!/bin/bash

cat << \
EOL
oh nos
/* this comment and everything
 * after has wrong colors
 */
EOL

function foo() {
    echo "Function foo"
}

foo

Without line break after <<

This works in VSCode and on Github.

Code:

#!/bin/bash

cat <<EOL
oh no's
/* this comment and everything
 * after has wrong colors
 */
EOL

function foo() {
    echo "Function foo"
}

foo

Originally from @NicolasGoeddel in microsoft/vscode#202493

[maxprehl]

I'm having this issue as well with multiline strings / heredocs that use the << operator.

I noticed that several characters break the highlighting.

I'd specifically like if the = equals sign was allowed as a character.

It's fairly common to use POD documentation in shell scripts by leveraging heredocs, and it's convenient to end the heredoc and the pod section with a single =cut.

Sources: wikipedia, 2007 blogpost

[maxprehl]

i took a stroll through the code, still not 100% on this, but i believe it comes down to this line and the similar ones in the following sections:

better-shell-syntax/main/main.rb

Line 959 in 7220287

:normal_statement_context,

All the heredocs: <<-"HEREDOC", <<"HEREDOC", <<-HEREDOC, <<HEREDOC use the :normal_statement_context grammar for the match on the HEREDOC name itself.

I'm no expert, but I can't think of a reason why this would need to be constrained to a "normal statement" the same way regular lines of shell script would be.

[jeff-hykin]

I can't think of a reason why this would need to be constrained to a "normal statement" the same way regular lines of shell script would be.

@maxprehl Thanks for looking through the source, thats always helpful for me. Unfortunately that code is needed because stuff like this is valid. (It highlights the grep howdy)

cat <<<'thing' | grep howdy
hello
Hi 
Howdy
thing

However the issue for =cut I think is actually a few lines above the source code you linked; the code expects an identifier inside of the single quotes instead of any possible string. That should be an easy fix.

The backslash problem at the top is not going to be an easy fix because Textmate parsers can only see one line at a time, and just use clever hacks to pretend to do multi-line parsing