Fix multiline strings and add document end marker support

[winstliu]

Requirements

• Filling out the template is required. Any pull request that does not include enough information to be reviewed in a timely manner may be closed at the maintainers' discretion.
• All new code requires tests to ensure against regressions

Description of the Change

YAML is tricky because it supports multiline strings using regular tags. As long as the next line is indented more than the tag, it'll be counted as part of the string. In previous PRs I implemented the multiline support, but without the indentation requirement. This PR properly implements indentation checks so that we don't attempt to tokenize everything until the next tag as scalar content. This fixes embedding (valid) YAML in other grammars, such as GFM.

In addition, the end document marker ... has been added.

Alternate Designs

None.

Benefits

The immediate benefits provided by this PR won't necessarily be evident to people who solely edit YAML because most non-tagged lines will be tokenized as a string anyway. However, it'll be extremely beneficial to grammars that allow for the embedding of YAML.

Possible Drawbacks

Not necessarily a drawback, but people don't get any indication that they're writing invalid YAML, even with this pull request. That is because all lines fall back to being tokenized a string if there are no other matches. That also makes it extremely hard to test, because the tags don't get an overarching scope name. So I've had to settle for a document end marker test, which will fail without indentation checks. This may be something that I'll try to fix for the future.

Applicable Issues

Fixes #72
Fixes #73

/cc @lierdakil

[lierdakil]

Thank you!

I can confirm that this fixes both issues.