Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Quoted::as_... doesn't consume all tokens #5803

Closed
asterite opened this issue Aug 22, 2024 · 3 comments · Fixed by #6064
Closed

Quoted::as_... doesn't consume all tokens #5803

asterite opened this issue Aug 22, 2024 · 3 comments · Fixed by #6064
Labels
bug Something isn't working

Comments

@asterite
Copy link
Collaborator

asterite commented Aug 22, 2024

Aim

When turning Quoted into something else, it seems once tokens were consumed to form the target type, the rest of the tokens are discarded. Maybe in that case it should error with "expecting EOF" or something like that.

For example:

fn main() {
    comptime
    {
        let expr = quote { x = hello world ([{}]) }.as_expr().unwrap();
        println(expr);
    }
}

Output:

x

Expected Behavior

The above program errors because the entire token stream doesn't parse to an expression.

Bug

Remaining tokens are silently discarded.

To Reproduce

Workaround

None

Workaround Description

No response

Additional Context

No response

Project Impact

None

Blocker Context

No response

Nargo Version

No response

NoirJS Version

No response

Proving Backend Tooling & Version

No response

Would you like to submit a PR for this Issue?

None

Support Needs

No response

@asterite asterite added the bug Something isn't working label Aug 22, 2024
@asterite asterite changed the title Quote::as_... doesn't consume all tokens Quoted::as_... doesn't consume all tokens Aug 22, 2024
@jfecher
Copy link
Contributor

jfecher commented Aug 22, 2024

We'll need to change the parsers these use to expect an EOF after the parser.

@asterite
Copy link
Collaborator Author

I'm also thinking, given that as_expr() can actually wrap a statement, an LValue, eventually a Pattern... maybe as_expr() would try to parse each of those things in turn, right? So you could do quote { a = 1; }.as_expr() instead of having to wrap that in a block expression.

@jfecher
Copy link
Contributor

jfecher commented Aug 23, 2024

Yeah. I do wonder if that could be problematic though. If there's ever when you'd parse a variable but you wanted an lvalue variable or a pattern variable or something. Not sure but if that's ever the case we'd have to split Expr into the actual 2-4 types it wraps.

github-merge-queue bot pushed a commit that referenced this issue Sep 17, 2024
# Description

## Problem\*

Resolves #5803
Resolves #6059

## Summary\*

If we ever had a token stream with a subset of valid input we'd parse
just that subset instead of erroring that the full input doesn't parse.

## Additional Context

We don't have EOF tokens in token streams so I used the `end()`
primitive instead.

## Documentation\*

Check one:
- [x] No documentation needed.
- [ ] Documentation included in this PR.
- [ ] **[For Experimental Features]** Documentation to be submitted in a
separate PR.

# PR Checklist\*

- [x] I have tested the changes locally.
- [x] I have formatted the changes with [Prettier](https://prettier.io/)
and/or `cargo fmt` on default settings.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
Status: Done
Development

Successfully merging a pull request may close this issue.

2 participants