macro_rules macros should not be serialized through strings in crate metadata.

[eddyb]

macro_rules should use TokenStreams instead of being pretty-printed into a string and later re-parsed. This is required to get correct/precise hygiene information and original source spans.

For example, right now macros from libstd get <std macros> in their spans instead of the original source, and we hide printing any of that source because it looks much worse than the original, and those spans also don't capture any fine-grained information from macro-defining macros.

IIRC @ibabushkin started working on this (see #43847) but I couldn't find a tracking issue.
cc @alexcrichton @jseyfried @Manishearth

[durka]

This should help with re-exported macro docs #26560 as well.

[nikomatsakis]

triage: P-medium

[nikomatsakis]

In case there is a decision to be made here, seems like clearly "yes", right? Otherwise, I'm not sure why this is nominated, so removing the tag.

[eddyb]

@nikomatsakis I think it was nominated to assign someone to it.

[ibabushkin]

Here is my last (rather old) attempt at this: branch, previous PR. I was unable to get this to a working state (due to lack of time, and strange issues).

[steveklabnik]

Triage: it's not clear to me where this lives in the compiler to see if it's something that has been fixed or not; does anyone know where to look? That would help future folks reading this ticket.

[petrochenkov]

#66364 fixes this for macro_rules (they are no longer encoded as strings).

Even after #66364 hygienic properties are still not encoded in metadata and are set heuristically on macro loading, that's one of the primary reasons why Span::def_site is not stabilized.

[eddyb]

I think merging #66364 should close this issue, because even though I said:

This is required to get correct/precise hygiene information and original source spans.

I mean that this is a prerequisite to precise hygiene information, but that is a separate concern that affect all serialized Spans and should be tracked separately.