macros should be parsed more lazily

[jbclements]

This is a FIXME issue; currently, macro invocations are parsed too eagerly. In particular, the current system decides before expansion time whether a rhs pattern-use (e.g. $y) is bound or unbound. This is incompatible with macro-producing macros that use a given identifier in a pattern position. It's also incompatible with macros inside macros, where inner binding patterns are believed to be pattern uses. The solution is to delay this decision until the macro is encountered at expansion-time, by which time all outer macros have been expanded.

Here's an example of a macro that doesn't work in the current system:

// the z flows into and out of two macros (g & f) along one path, and one (just g) along the
// other, so the result of the whole thing should be "let z_123 = 3; z_123
macro_rules! g (
    ($x:ident) =>
    (
        {macro_rules! f(
            ($y:ident)=>({let $y=3;$x})
        );
         f!($x)})
)

fn a(){g!(z)}

Trying to expand this yields this error:

Running /usr/local/bin/rustc:
/tmp/g.rs:7:12: 7:13 error: unknown macro variable `y`
/tmp/g.rs:7             ($y:ident)=>({let $y=3;$x})
                        ^

... indicating that the $y is believed to be a use, rather than a binder.

[Aatch]

Nominating for well-defined.

[bblum]

double triage before triage meeting. i think this could maybe be well-covered; we understand how it should work, and it's not a backwards-compat hazard.

[kud1ing]

I think the tag "A-macos" is unfortunate, since this is not Mac OS specific. Or is it?

[huonw]

I imagine it's just a typo because macos is almost macros. It reproduces on linux. (Removed that tag.)

[kud1ing]

I guess so. Maybe rename "A-macos" to "A-mac_os" or "A-mac"?

[catamorphism]

Accepted for milestone 3, feature-complete

[jbclements]

Wow! glad to see this wind up on a milestone...

[pnkfelix]

P-low, not 1.0.

[huonw]

Would this be fixable by allowing $ escapes? E.g.

// the z flows into and out of two macros (g & f) along one path, and one (just g) along the
// other, so the result of the whole thing should be "let z_123 = 3; z_123
macro_rules! g (
    ($x:ident) =>
    (
        {macro_rules! f(
            ($$y:ident)=>({let $$y=3;$x})
        );
         f!($x)})
)

fn a(){g!(z)}

The $$'s would "expand" to a single $ in the expanded code.

[jbclements]

No, don't go down that road; I don't believe that will lead to a nice solution.

[huonw]

OK.

It would be nice if the eventual solution was not specific to nested macro_rules!, since one may wish to have a macro like e.g. foo (($e: expr) => { quote_expr!(cx, $$a + $e) }) (using the escaping for clarity), that is, substitute the passed in expression directly, but pass in a literal $a for quote_expr to handle internally... foo!(2) should become quote_expr!(cx, $a + 2).

(I could easily imagine other more common procedural macros that may wish to "overload" the $... syntax, e.g. a sql!(SELECT * FROM foo WHERE bar = $x) macro.)

[jbclements]

Hang on... where is the $a bound, in your example?

[huonw]

Sorry, was on my phone making it difficult to explain fully. I meant something like

let a: i32 = ...;

macro_rules! foo {
    ($e: expr) => { quote_expr!(cx, $$a + $e) }
}

foo!(1);
foo!(2);

The above is theoretically equivalent to the following, which is unquoting/splicing a into the Gc<syntax::ast::Expr> that quote_expr returns

let a: i32 = ...;
quote_expr!(cx, $a + 1);
quote_expr!(cx, $a + 2);

(The let a: i32 = ...; could also be scoped inside the macro definition.)

[jbclements]

in this case, you shouldn't need $s at all; the free variable a should always refer to the environment of the definition of the macro.

[jbclements]

I'm thinking more about this issue, and I think it's misnamed. The fundamental issue here is how the transcription engine matches up bindings of macro variables with uses of those variables.

The solution, I think, is pretty straightforward: the transcription engine should just ignore transcription
variables that it doesn't recognize.

Also, hygiene should stick its nose in, here. There should be a renaming of all pattern identifiers. I think the sneaky side-effect of this will be to make the $'s in the right-hand sides redundant.

[huonw]

in this case, you shouldn't need $s at all; the free variable a should always refer to the environment of the definition of the macro.

No, quote_expr!(cx, a + 1) is very different to quote_expr!(cx, $a + 1). Simillarly, sql!(SELECT * FROM table WHERE foo = a) is different to sql!(SELECT * FROM table WHERE foo = $a).

The ones with $ are inserting the value of a from the program with the macro, i.e. if let a = 1i32; then the $a is the same as writing 1. The ones without a are just using a as an identifier (for the quoting, it becomes an expression representing Ident("a") + Literal(1); for sql, it's filtering for the foo column equalling the a column).

[jbclements]

Hey! Let's argue! :)

I understand now better what you're looking for. Specifically, you want to use $ as an unquote mechanism in procedural macros, in order to splice in other syntactic terms and values from the context of the macro definition (specifically, I missed the fact that you were using quote_expr! before, sorry). This is plausible, but I'd like to point out that it's not the same as what $a does currently—it's not an unquoting mechanism. This is perhaps most clear when the identifier 'a' is bound to a sequence rather than a single syntactic term, but even in the single-match case, the RHS use of $a is not actually inserting the value of an 'a' variable defined in a surrounding context.

I think that something like this is possible, but I don't think that re-using $ is the right way to go.

Lemme fix this, and then we'll see where we are.

[jbclements]

Oh dear Lord... I may have spoken too quickly. Or perhaps I meant to say, I may have spoken way too late. I see that the source is already peppered with uses of $ that may in fact be unquotes. Reading more...

[jbclements]

Yes, I see that it absolutely is used that way. That's a bad pun, and I think it might be painful....

[huonw]

The problem would be resolved IMO if we have some technique that works for those cases that's not $variablename. (That said, $ is a nice-ish character, so having it as flexible as possible would be nice, but obviously not required.)

[steveklabnik]

Triage: OP's code, with a semicolon, still generates the same error.

[bltavares]

Triage: There is still the same error happening on rustc 1.11.0-nightly (6e00b5556 2016-05-29)

// the z flows into and out of two macros (g & f) along one path, and one (just g) along the
// other, so the result of the whole thing should be "let z_123 = 3; z_123
macro_rules! g (
    ($x:ident) =>
    (
        {macro_rules! f(
            ($y:ident)=>({let $y=3;$x})
        );
         f!($x)})
);

fn a(){g!(z)}

Error:

/tmp/zshKZx8Ym:8:14: 8:22 error: unknown macro variable `y`
/tmp/zshKZx8Ym:8             ($y:ident)=>({let $y=3;$x})
                              ^~~~~~~~
/tmp/zshKZx8Ym:8:31: 8:33 error: unknown macro variable `y`
/tmp/zshKZx8Ym:8             ($y:ident)=>({let $y=3;$x})
                                               ^~
error: aborting due to 2 previous errors

[B1Z0N]

It seem to be working poorly on variadic arguments. Or am I missing something?

CODE

Here

macro_rules! code_macro {
  ($gen:ident, $res:ident) => {
    macro_rules! $res {
      ($($token:tt)*) => {
        http_code!($gen $($token)*)
      };
    }
  };
  ($closure:tt, $res:ident) => {
    let f = $closure;
    code_macro!(f, $res)
  };
}