Don't highlight # which does not start an attribute in rustdoc

[mystor]

Currently when we highlight some macros for rustdoc (e.g. quote! from https://github.com/dtolnay/quote), we get really bad syntax highlighting, because we assume that every token between a # character and the next ] in the source must be an attribute.

This patch improves that highlighting behavior to instead only highlight after finding the [ token after the # token.

(NOTE: I've only run this patch against https://github.com/nrc/rustdoc-highlight so if it doesn't build on travis that's why - I don't have a recent rustc build on this laptop)

I'm guessing r? @steveklabnik

[mystor]

See the docs for synstructure for how bad the current highlighting can look (https://docs.rs/synstructure/0.5.2/synstructure/)

[QuietMisdreavus]

Looks like the travis failure was in running this test, which was put in by #41783 and #41785. The test expects the behavior you're trying to change, so it may be prudent to change the test? I'm not too sure on the particulars.

r? @GuillaumeGomez

[GuillaumeGomez]

I would actually be a bit more strict on the check: "if the character after # isn't a whitespace, then it shouldn't be highlighted". What do you think of this?

[carols10cents]

Hi @mystor! friendly ping to keep this on your radar! Wdyt of @GuillaumeGomez's suggestion?

[mystor]

Sorry! I have been very busy over the last week-or-so due to travel for RustConf and then the subsequent catching up on work, so this patch slipped my mind.

I don't think I fully understand what @GuillaumeGomez is suggesting. From what I can tell, the test referenced by @QuietMisdreavus was testing hiding a line from rustdoc output with the # character.

Namely, currently the following happens:

# This will be hidden
# # This will also be hidden (note:  the space)
## This escapes the `#` at the beginnig of the line and has one # character
### This only escapes the first `#` character.
#### Again, the same.

The reason why this change affects this test was because when we read a non-escaped # character in the code highlighter we began an attribute, whether or not it was syntactically the start of an attribute. You can actually see that in the expected test output. That output was expected to produce partial HTML output, as it never parses a ] character to terminate the attribute.

With this change, none of these lines would be indented, as none of them form a valid attribute. A line like:

##[This is an attribute]

would be parsed as an attribute, and would have a complete attribute span tag around it.

[QuietMisdreavus]

@GuillaumeGomez Wouldn't this just take out the attribute highlighting altogether? Or are you wanting to highlight # characters by themselves? I like the way this PR does it, since it specifically checks that something looks like an attribute before highlighting it as such. Otherwise it just doesn't do anything special.

[GuillaumeGomez]

No, it's fine as is.

[QuietMisdreavus]

Cool! This looks good, thanks for the PR!

@bors r+

[bors]

📌 Commit 2f19383 has been approved by QuietMisdreavus