Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(parser): Update lexer methods to handle only ASCII identifiers #1558

Merged
merged 1 commit into from
Jan 15, 2024

Conversation

denbezrukov
Copy link
Contributor

Summary

We don't need to track non-ASCII chars since all our keywords are ASCII.

Adjusted the methods consume_ident_sequence and consume_and_get_ident in the biome_css_parser's and biome_js_parser's lexer modules, to return an additional boolean indicating whether only ASCII characters were used. This change will improve handling of non-ASCII identifiers in CSS and JS parsing, providing a more accurate classification of identifiers.

Test Plan

CI

Copy link

netlify bot commented Jan 13, 2024

Deploy Preview for biomejs ready!

Name Link
🔨 Latest commit aca8cb4
🔍 Latest deploy log https://app.netlify.com/sites/biomejs/deploys/65a500cf359e2d0008eeb50b
😎 Deploy Preview https://deploy-preview-1558--biomejs.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.
Lighthouse
Lighthouse
1 paths audited
Performance: 94 (🔴 down 5 from production)
Accessibility: 97 (no change from production)
Best Practices: 100 (no change from production)
SEO: 93 (no change from production)
PWA: -
View the detailed breakdown and full score reports

To edit notification comments on pull requests, go to your Netlify site configuration.

@github-actions github-actions bot added A-Parser Area: parser L-JavaScript Language: JavaScript and super languages L-CSS Language: CSS labels Jan 13, 2024
Copy link
Contributor

Parser conformance results on

js/262

Test result main count This PR count Difference
Total 49701 49701 0
Passed 48721 48721 0
Failed 980 980 0
Panics 0 0 0
Coverage 98.03% 98.03% 0.00%

jsx/babel

Test result main count This PR count Difference
Total 40 40 0
Passed 37 37 0
Failed 3 3 0
Panics 0 0 0
Coverage 92.50% 92.50% 0.00%

symbols/microsoft

Test result main count This PR count Difference
Total 6322 6322 0
Passed 2036 2036 0
Failed 4286 4286 0
Panics 0 0 0
Coverage 32.20% 32.20% 0.00%

ts/babel

Test result main count This PR count Difference
Total 662 662 0
Passed 592 592 0
Failed 70 70 0
Panics 0 0 0
Coverage 89.43% 89.43% 0.00%

ts/microsoft

Test result main count This PR count Difference
Total 17646 17646 0
Passed 13452 13452 0
Failed 4192 4192 0
Panics 2 2 0
Coverage 76.23% 76.23% 0.00%

Copy link

codspeed-hq bot commented Jan 13, 2024

CodSpeed Performance Report

Merging #1558 will not alter performance

⚠️ No base runs were found

Falling back to comparing refactor/lexer-ascii (aca8cb4) with main (593da22)

Summary

✅ 93 untouched benchmarks

Adjusted the methods `consume_ident_sequence` in the biome_css_parser's lexer modules, to return an additional boolean indicating whether only ASCII characters were used. This change will improve handling of non-ASCII identifiers in CSS, providing a more accurate classification of identifiers.
@denbezrukov denbezrukov merged commit 3a279b6 into main Jan 15, 2024
21 checks passed
@denbezrukov denbezrukov deleted the refactor/lexer-ascii branch January 15, 2024 10:52
ematipico pushed a commit to DaniGuardiola/biome that referenced this pull request Jan 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-Parser Area: parser L-CSS Language: CSS L-JavaScript Language: JavaScript and super languages
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants