-
-
Notifications
You must be signed in to change notification settings - Fork 34
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Create complete tests for syntax #843
Labels
blocker-candidate
The submitter thinks this might be a block for the Technology Preview
LDML46.1
MF2.0 Draft Candidate
test-suite
Comments
catamorphism
added a commit
to catamorphism/message-format-wg
that referenced
this issue
Jul 30, 2024
catamorphism
added a commit
to catamorphism/message-format-wg
that referenced
this issue
Jul 30, 2024
Doing much better than #844 would probably require some automation. |
Note that completeness also requires exhaustively generating tests that should produce syntax errors, which I have less of an idea how to do systematically (other than through fuzzing.) |
I wrote a simple random test generator, hard-wired for the MF2 grammar, and was able to find some bugs in the ICU4C parser with the generated test cases. |
catamorphism
added a commit
to catamorphism/message-format-wg
that referenced
this issue
Jul 31, 2024
catamorphism
added a commit
to catamorphism/message-format-wg
that referenced
this issue
Jul 31, 2024
catamorphism
added a commit
to catamorphism/message-format-wg
that referenced
this issue
Jul 31, 2024
catamorphism
added a commit
that referenced
this issue
Aug 5, 2024
) * Add syntax tests for unsupported expressions and statements (#843) * Update test/tests/unsupported-statements.json Co-authored-by: Eemeli Aro <[email protected]> --------- Co-authored-by: Eemeli Aro <[email protected]>
catamorphism
added a commit
that referenced
this issue
Aug 5, 2024
aphillips
added
blocker-candidate
The submitter thinks this might be a block for the Technology Preview
LDML46.1
MF2.0 Draft Candidate
labels
Nov 16, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
blocker-candidate
The submitter thinks this might be a block for the Technology Preview
LDML46.1
MF2.0 Draft Candidate
test-suite
This is from https://github.com/unicode-org/message-format-wg/wiki/Things-That-Need-Doing . I didn't see an existing issue for it.
Full test coverage is probably impossible given the size of the grammar. But there are a couple possibilities:
As far as option 2, with a few modifications, I was able to run the grammar through abnfgen and Eusthasius. The problem is that both of them generate tests that resemble line noise, suggesting that fuzzing might be preferable.
Option 1 has the advantage that it's easy to write readable, minimal test cases. Of course the disadvantage is that it's tedious.
Any suggestions are welcome.
The text was updated successfully, but these errors were encountered: