-
Notifications
You must be signed in to change notification settings - Fork 16
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
SmaCC_Elixir: parsing expr causes token after <at_op>, unary_op, or <capture_op> dropping next identifier #36
Comments
I've made some progress, after staring at it for a long time, I did properly get it to error, however it doesn't work in general as I have some errors when throwing big files at this change actionsForCurrentToken
| lookahead ids id actions token position previousState |
scanner lastWasEol
ifTrue: [ actions := OrderedCollection new.
self addActionsFor: scanner eolId to: actions.
actions notEmpty
ifTrue: [ scanner lastWasEol: false.
position := scanner position.
token := currentToken.
previousState := currentState.
currentToken := currentToken class
value: ''
start: token startPosition
ids: {scanner eolId}.
currentState := self duplicateState.
self position: position -3.
self parseCurrentToken.
currentToken := token.
self position: position.
self parseCurrentToken.
self restoreState: previousState.
self position: position.
scanner lastWasEol: true.
currentToken := token ] ].
I've added all the Here is the current state after this change on some definition |
I believe I have a solution that I will open in a PR. However I'm not sure what I'm doing precisely by this: actionsForCurrentToken
| lookahead ids id actions token position previousState |
scanner lastWasEol
ifTrue: [ actions := OrderedCollection new.
self addActionsFor: scanner eolId to: actions.
actions notEmpty
ifTrue: [ scanner lastWasEol: false.
position := scanner position.
token := currentToken.
previousState := currentState.
currentToken := currentToken class
value: ''
start: token startPosition
ids: {scanner eolId}.
currentState := self duplicateState.
self position: position - token size.
self parseCurrentToken.
currentToken := token.
self position: position.
self parseCurrentToken.
self restoreState: previousState.
self position: position.
scanner lastWasEol: true.
currentToken := token ] ]. Seems to work, I deduced this by noting that the number I go This was gotten by looking at how |
The Position wasn't being shifted before scanning forward unlike the map_op case. We shifft it by the characters to remove the issue.
Issue
Parsing multiple Expressions
Has issues on the two parts of grammar below
The next parsed item drops the next identifier, unless if the next identifier is required for a valid following expression.
I believe this may be related to
glr
backtracking andactionsForCurrentToken
. As changingexpr
to eitherunmatched_expr
ormatched_expr
seems to create correct parses for their respective cases, and trying it out in thegrammar
tab (which seems not to be affected byactionsForCurrentToken
) returns the proper value.I'm not sure of the issue, as the scanner state is properly restored(?).
I.E.
will properly display both expressions where as.
will only have
&3
and()
as the expressions.As a consequence, the following parses, when the following
)
should be had.Example with
<at_op>
the
def
is missing and thenock( ...
is next.Example with
<capture_op>
The foo is missing and the next token is
()
Example with
unary_op
The next expression is
1
Analysis
It seems that putting the operator within a expression does not make a difference
will be treated as if it were
However, Putting a
;
between the nodes make it work, as perThis may be related to #29, as the
<unary_ops>
seem to trigger similar errorsDebugging Information
To better debug what is going on I started to compare
The parser should mostly treat these the same, However, after the
expr_list
is made for both, the first one back tracks to be anunmatched_expression
which is the next rule aftermatched_expression
. Seems theglr
back tracking kicks in.One step later
The text was updated successfully, but these errors were encountered: