-
-
Notifications
You must be signed in to change notification settings - Fork 475
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
refactor(parser): Update lexer methods to handle only ASCII identifiers #1558
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
✅ Deploy Preview for biomejs ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
github-actions
bot
added
A-Parser
Area: parser
L-JavaScript
Language: JavaScript and super languages
L-CSS
Language: CSS
labels
Jan 13, 2024
Parser conformance results onjs/262
jsx/babel
symbols/microsoft
ts/babel
ts/microsoft
|
CodSpeed Performance ReportMerging #1558 will not alter performanceFalling back to comparing Summary
|
ematipico
reviewed
Jan 13, 2024
denbezrukov
force-pushed
the
refactor/lexer-ascii
branch
from
January 14, 2024 20:15
90bc96b
to
4a7ad59
Compare
Adjusted the methods `consume_ident_sequence` in the biome_css_parser's lexer modules, to return an additional boolean indicating whether only ASCII characters were used. This change will improve handling of non-ASCII identifiers in CSS, providing a more accurate classification of identifiers.
denbezrukov
force-pushed
the
refactor/lexer-ascii
branch
from
January 15, 2024 09:54
4a7ad59
to
aca8cb4
Compare
ematipico
pushed a commit
to DaniGuardiola/biome
that referenced
this pull request
Jan 24, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
We don't need to track non-ASCII chars since all our keywords are ASCII.
Adjusted the methods
consume_ident_sequence
andconsume_and_get_ident
in the biome_css_parser's and biome_js_parser's lexer modules, to return an additional boolean indicating whether only ASCII characters were used. This change will improve handling of non-ASCII identifiers in CSS and JS parsing, providing a more accurate classification of identifiers.Test Plan
CI