Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

WebCryptoAPI: Use .any.js for generateKey tests and split up slow tests #10984

Merged
merged 3 commits into from
May 15, 2018
Merged

WebCryptoAPI: Use .any.js for generateKey tests and split up slow tests #10984

merged 3 commits into from
May 15, 2018

Conversation

zcorpan
Copy link
Member

@zcorpan zcorpan commented May 14, 2018 •
edited by wpt-pr-bot
Loading

Slow tests test 10 subtests at a time, using the // META: variant=...
feature.

Fixes #10794.

@zcorpan zcorpan requested a review from Hexcles May 14, 2018 13:03
@zcorpan zcorpan force-pushed the zcorpan/generateKey-split-slow-tests branch from 305131f to 19f8aab Compare May 14, 2018 13:13
@zcorpan
Copy link
Member Author

zcorpan commented May 14, 2018

This needs to be rebased on top of #10985 to satisfy the linter.

@Hexcles
Copy link
Member

Hexcles commented May 14, 2018

I like the use of .any.js.

The splitting is done with variants. The indexes are hard coded, which might be a bit brittle. I'm not sure how to improve though. One idea I have is to not hard code the last index, but use something like 150-last instead to run all the remaining subtests, which also has the downside of potentially making the last split too large if people don't pay attention when adding more subtests later.

@zcorpan
WebCryptoAPI: Use .any.js for generateKey tests and split up slow tests
e2bae53 
Slow tests test 10 subtests at a time, using the `// META: variant=...`
feature.

Fixes #10794.
@zcorpan zcorpan force-pushed the zcorpan/generateKey-split-slow-tests branch from 19f8aab to e2bae53 Compare May 14, 2018 15:07
@zcorpan
Copy link
Member Author

zcorpan commented May 14, 2018

Thanks, good idea about last, now implemented.

@jimsch
Copy link
Contributor

jimsch commented May 14, 2018

@zcorpan Do we have any idea if there are more or fewer failures than before. I am not seeing anything in the changes that triggers to me that we will have more failures, but trying to look through the travis logs it seems that there are a great number of fails.

@zcorpan
Copy link
Member Author

zcorpan commented May 15, 2018

Comparing https://w3c-test.org/WebCryptoAPI/generateKey/ and https://w3c-test.org/submissions/10984/WebCryptoAPI/generateKey/

Test Chrome, master Chrome, this PR Firefox, master Firefox, this PR
failures_AES-CBC 736 Pass 2 Fail 736 Pass 2 Fail 738 Fail 738 Fail
successes_RSA-OAEP 156 Pass 156 Pass 156 Pass 156 Pass
successes_RSA-PSS 36 Pass 36 Pass 36 Pass 36 Pass
successes_RSASSA-PKCS1-v1_5 36 Pass 36 Pass 36 Pass 36 Pass

(This is unfortunately manual collection of results, see #7475)

In Firefox, the all-fail in failures_AES-CBC is because of assert_equals: Bad algorithm not supported expected "NotSupportedError" but got "SyntaxError" i.e. wrong exception being thrown.

jimsch
jimsch reviewed May 15, 2018
View reviewed changes
WebCryptoAPI/generateKey/successes.js Outdated
var match;
if (location.search) {
match = /^\?(\d+)-(\d+|last)$/.exec(location.search);
subTestStart = match[1];
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if the match operation fails here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made it run all tests.

jimsch
jimsch approved these changes May 15, 2018
View reviewed changes
Copy link
Contributor

@jimsch jimsch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Things look good. Thanks for the change

@zcorpan zcorpan merged commit b395fb0 into master May 15, 2018
@zcorpan zcorpan deleted the zcorpan/generateKey-split-slow-tests branch May 15, 2018 17:45
@zcorpan
Copy link
Member Author

zcorpan commented May 15, 2018

Thanks for the reviews!

@zcorpan zcorpan mentioned this pull request May 15, 2018
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jimsch jimsch jimsch approved these changes

@Hexcles Hexcles Awaiting requested review from Hexcles

@engelke engelke Awaiting requested review from engelke

@wafflespeanut wafflespeanut Awaiting requested review from wafflespeanut

Assignees
No one assigned
Labels
WebCryptoAPI wg-webcrypto
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@zcorpan @Hexcles @jimsch @wpt-pr-bot