Enable finding the set of tests that pass in 3 or greater browsers

[lukebjerring]

From @jdm on October 3, 2017 13:16

Servo is a big consumer of web-platform-tests but doesn't really warrant inclusion as a data point (due to instability, lack of meaningful audience, etc.) However, it would be useful to be able to see the set of tests that are passing in most other browsers, and to do so in a way that is easy to consume in a scripted fashion (eg. JSON). Historical data for other browsers is not particularly interesting to us; we're only interested in a snapshot on any given day so we can compare our own test results against the latest data from other browsers.

Copied from original issue: web-platform-tests/results-collection#135

[lukebjerring]

From @foolip on October 3, 2017 16:52

This kind of filtering is a topic of web-platform-tests/results-collection#83, but having a JSON endpoint isn't part of the plan so far. Would web-platform-tests/results-collection#136 solve the problem if it's possible to just download the latest results instead of everything?

Actually, if you run wpt.fyi it will download some data files that may be exactly what you need, I see https://storage.googleapis.com/wptd/85a171624e/chrome-62.0-linux-summary.json.gz and others in developer tools. Can you see if that has the data you need, and if a nicer endpoint for just that might suffice?

[lukebjerring]

From @jdm on November 1, 2017 19:55

Yes, that data looks like it's thorough enough for what we're looking for.

[lukebjerring]

From @jgraham on November 1, 2017 20:5

Note that that data is per-file rather than per-subtest so it's generally not possible to tell if each browser is passing the same subset of the tests except in trivial cases (e.g. all subtests have the same behaviour).