Make it possible to run tests from file:///

[gsnedders]

This is something that has come up before, and came up again in #10053. It's come up various times before, mostly from WebKit people. So, for the sake of unifying discussion, here's one issue to rule them all.

This means:

• No absolute paths in tests.
• No referring to files further up the directory tree (given this violates the SOP for file:/// in some UAs).
• No referring to a least some file types further down the directory tree (given this violates the SOP for file:/// in some UAs). (Though this would break support files in the CSS build system output, as they're required to be in a support directory adjacent to the test.)

cc/ @jgraham @foolip @youennf @fantasai @dbaron @mrego

[dbaron]

So I think it's worth clearly distinguishing issues related to the same-origin-policy (affecting fonts) from issues affecting the ability to load resources at all (inability to load things that can be loaded cross-origin, like scripts, style sheets, and images).

In Gecko, different directories are considered separate origins, so you can't load downloadable fonts (which are subject to the same origin policy) from a different directory (including a subdirectory). I'm not aware of within-file: restrictions on loading in Gecko, though I'm aware of other restrictions (e.g., loading file: from an http: origin), but my information may well be out-of-date.

I'm not familiar with UAs that restrict script/style/images up a directory. I'd be curious to hear more details -- but I also feel like making things work for file: URLs in such UAs may impose too many constraints; things that follow normal development practices and work on file: URLs in most UAs often won't work in such UAs.

[gsnedders]

Okay, it might only be the SOP that's affected by going up a directory and not anything else. I had the impression from somewhere it was more general.

That said, there seems to be even less interop when it comes to the SOP on file:/// than I thought there was, which makes it awkward if we want to make it possible to run tests from file:/// (especially given it's undefined and UAs have slowly been making the SOP on file:/// increasingly restrictive).

[jgraham]

I am pretty strongly aganst this. Basing interoperability testing on an intentionally non-interoperable part of the platform where the rules are not well understood and are subject to change at any moment seems extraodinarily ill-advised.

This would also impose significant burden on test authors that typically don't exist for proprietary tests (even when these are run over file:// one generally only has to care about a single implementation, and if that implementation changes the tests can be changed without fear of breakage elsewhere). Imposing unneessary requirements that make wpt unergonomic to author compared to the alterntaives is something we have very intentionally tried to avoid since historically we see that resulting in people chosing not to write shared tests.

[foolip]

I'm also not very optimistic that we could make this work. But I don't think it's really a policy question, what we need is a concrete proposal that someone would be willing to drive. Would the goal be to ensure that all tests can run over file: URLs and enforce it using lints, or just to allow some tests to be run over file: where there is no real dependency on an HTTP server? Without more specifics, it's hard to say if this is a good idea or not.

[guest271314]

What would be helpful is not only the ability to run one or more tests at file: protocol, but also the ability to download/clone/upload only specific tests; given that this repository was over 200MB, when last checked #9918.