Skip to content
This repository has been archived by the owner on Nov 6, 2019. It is now read-only.

Enable finding the set of tests that pass in 3 or greater browsers #135

Closed
jdm opened this issue Oct 3, 2017 · 4 comments
Closed

Enable finding the set of tests that pass in 3 or greater browsers #135

jdm opened this issue Oct 3, 2017 · 4 comments

Comments

@jdm
Copy link

jdm commented Oct 3, 2017

Servo is a big consumer of web-platform-tests but doesn't really warrant inclusion as a data point (due to instability, lack of meaningful audience, etc.) However, it would be useful to be able to see the set of tests that are passing in most other browsers, and to do so in a way that is easy to consume in a scripted fashion (eg. JSON). Historical data for other browsers is not particularly interesting to us; we're only interested in a snapshot on any given day so we can compare our own test results against the latest data from other browsers.

@foolip
Copy link
Member

foolip commented Oct 3, 2017

This kind of filtering is a topic of #83, but having a JSON endpoint isn't part of the plan so far. Would #136 solve the problem if it's possible to just download the latest results instead of everything?

Actually, if you run wpt.fyi it will download some data files that may be exactly what you need, I see https://storage.googleapis.com/wptd/85a171624e/chrome-62.0-linux-summary.json.gz and others in developer tools. Can you see if that has the data you need, and if a nicer endpoint for just that might suffice?

@jdm
Copy link
Author

jdm commented Nov 1, 2017

Yes, that data looks like it's thorough enough for what we're looking for.

@jgraham
Copy link
Collaborator

jgraham commented Nov 1, 2017

Note that that data is per-file rather than per-subtest so it's generally not possible to tell if each browser is passing the same subset of the tests except in trivial cases (e.g. all subtests have the same behaviour).

@lukebjerring
Copy link
Collaborator

This issue was moved to web-platform-tests/wpt.fyi#28

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

5 participants