Compiling a Rocket application on the 2015 edition doesn't emit errors

[ferrohd]

When trying to compile any Rocket application in the 2015 edition the compiler says an error occured without showing any actual error:

$ RUST_BACKTRACE=1 cargo build
   Compiling repro v0.1.0
error: could not compile `repro`
$

Switching to the 2018 edition fixes the problem, by either showing errors or properly compiling the application. Minimal reproduction:

// src/main.rs

#[rocket::launch]
fn foo() -> _ {
    rocket::build()
}

# Cargo.toml

[package]
name = "repro"
version = "0.1.0"
edition = "2015"

[dependencies]
rocket = "0.5.0-rc.1"

Meta

rustc --version --verbose:

rustc 1.55.0 (c8dfcfe04 2021-09-06)
binary: rustc
commit-hash: c8dfcfe046a7680554bf4eb612bad840e7631c4b
commit-date: 2021-09-06
host: x86_64-unknown-linux-gnu
release: 1.55.0
LLVM version: 12.0.1

[hkratz]

This leads to FatalError::raise() in the parser, which apparently does not produce any output nor tracing.

@rustbot label +I-ICE

[hkratz]

The fatal error is raised here:

rust/compiler/rustc_parse/src/parser/mod.rs

Line 527 in 15491d7

FatalError.raise();

[ehuss]

Here's an overview of what is happening.

The proc-macro creates something along the lines of:

fn main() { ::rocket::async_main(async move { /* stuff */ }) }

In 2015, when it goes to parse the arguments, it sees async as a normal identifier. It then sees move which is unexpected, which creates an unexpected error in the parser. However, here it attempts to recover by assuming the user forgot a ,. The parser then sees move and assumes that must be the start of a closure. However, it then sees { when it expected a | to start the closure arguments. This emits another unexpected error. However, this hits the FatalError here because it believes it has not made any progress. The original error has not yet been emitted, and the compiler exits without a message.

Normally this sequence of events won't happen because the last_unepxected_token_span would be different, and it would just emit a normal error. However, rocket is using quote_spanned! which causes all of the tokens to end up with the same span.

I think the use of FatalError.raise in expect_one_of is questionable, and not really explained in the code why it is there. Maybe this is to prevent the parser from getting stuck somehow? That could really use some comments explaining its purpose. Without understanding the purpose there, it is hard to guess at a possible fix.

Here is a minimal repro:

// src/lib.rs
#[pm::foo]
fn foo() {}

And a proc-macro named pm:

extern crate proc_macro;
use proc_macro::*;

#[proc_macro_attribute]
pub fn foo(_attr: TokenStream, item: TokenStream) -> TokenStream {
    let tt = item.into_iter().next().unwrap();
    let sp = tt.span();
    let mut arg = TokenStream::new();
    let mut g = Group::new(Delimiter::Brace, TokenStream::new());
    g.set_span(sp);
    arg.extend([
        TokenTree::Ident(Ident::new("async", sp)),
        TokenTree::Ident(Ident::new("move", sp)),
        TokenTree::Group(g),
    ]);
    let mut body = TokenStream::new();
    body.extend([
        TokenTree::Ident(Ident::new("async_main", sp)),
        TokenTree::Group(Group::new(Delimiter::Parenthesis, arg)),
    ]);

    let mut ret = TokenStream::new();
    ret.extend([
        TokenTree::Ident(Ident::new("fn", sp)),
        TokenTree::Ident(Ident::new("main", sp)),
        TokenTree::Group(Group::new(Delimiter::Parenthesis, TokenStream::new())),
        TokenTree::Group(Group::new(Delimiter::Brace, body)),
    ]);
    eprintln!("{:?}", ret);
    ret
}

[ehuss]

That check was added in #58903, but unfortunately there isn't much of a description explaining why it was added.

[estebank]

IIRC I believe that it was to deduplicate some redundant errors that occurred otherwise. This whole ordeal reminds me that we should be trying to parse async blocks in 2015 when encountering the kw in idents and then encountering parse errors to emit clear diagnostics about setting the edition (I thought we were doing that already). Macros can also set the edition metadata in the Span, IIRC (it might be an unstable API), which can let 2018 proc-macros work even in 2015 crates (do we even want to open that can of worms, I don't think so). I'm glad to see this is a parser error, it means that we can fix it "easily".

[Enselic]

Triage: Can someone please give a concrete example of a minimal "Rocket application" that reproduces this?

[ehuss]

@Enselic Is there any particular reason you'd like a Rocket application specifically? There is a minimal reproduction without Rocket listed above at #89699 (comment).

I believe this is mostly addressed via #114237. The reproduction now prints the error:

error: `async move` blocks are only allowed in Rust 2018 or later

However, I would suggest someone adding that reproduction to the UI testsuite, since it is a relatively subtle construction of tokens, and the test in #114237 doesn't test anything like what that is doing.

I'd also defer to @estebank if they think there is anything more fundamental that should be done, particularly around the use of last_unepxected_token_span and FatalError in expect_one_of. I'm a bit uneasy about the use of FatalError there, since it is capable of stopping the compiler even if no errors have been emitted. Similarly, comparing spans against last_unepxected_token_span may not always work if something like a proc-macro ends up using the same span across multiple tokens. I would imagine someone could cook up a different example that has the same problem where the compiler exits with no error.

[Enselic]

My bad, sorry about that. I'm trying to keep a high pace when triaging ICE-issues since there are so many of them. In this case the pace was too high, so I missed that comment of yours.

[vincenzopalazzo]

We discuss this in the wg-macros triage meeting and @CohenArthur will work on it 🛩️

If you want to join the discussion, please join our zulip stream

[CohenArthur]

@rustbot claim