Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: revert mjs extension usage by default, make it an option #9179

Merged
merged 12 commits into from
Feb 25, 2023

Conversation

dummdidumm
Copy link
Member

fixes #9141

hides the .mjs stuff behind an option which is false by default (else it's a breaking change, strictly speaking). I chose output.mjs as the option name. If we keep that, it may make sense to move outDir (top level currently) into that for version 2.

Please don't delete this checklist! Before submitting the PR, please make sure you do the following:

  • It's really useful if your PR references an issue where it is discussed ahead of time. In many cases, features are absent for a reason. For large changes, please create an RFC: https://github.com/sveltejs/rfcs
  • This message body should clearly illustrate what problems it solves.
  • Ideally, include a test that fails without this PR but passes with it.

Tests

  • Run the tests with pnpm test and lint the project with pnpm lint and pnpm check

Changesets

  • If your PR makes a change that should be noted in one or more packages' changelogs, generate a changeset by running pnpm changeset and following the prompts. Changesets that add features should be minor and those that fix bugs should be patch. Please prefix changeset messages with feat:, fix:, or chore:.

@Conduitry
Copy link
Member

The purist in me wants two changesets for this, one (a fix) rolling back the previous change, and another (a feature) adding the flag.

@benmccann
Copy link
Member

Do we need to add it to https://kit.svelte.dev/docs/configuration or is that auto-generated?

@dummdidumm
Copy link
Member Author

Autogenerated

@Rich-Harris
Copy link
Member

Rolling it back will hurt performance for the vast majority of cases where .mjs is supported. Should we add an option to use .js instead that defaults to false?

I recognise that there are semver arguments in favour of rolling it back, but I think it's worth considering impact.

@benmccann
Copy link
Member

I'd stick with .js by default. It's more important that it works than that it's fast. Also, while it will take awhile to roll out and for users to upgrade, WebKit just merged modulepreload support yesterday, which will make the .mjs trick obsolete.

@Rich-Harris
Copy link
Member

It's more important that it works than that it's fast

There's nuance here though. Suppose .mjs works for 99% of apps, and 15% of users are on Safari — that means that we're penalising those 15% of users, not for the sake of 1% of users but 1% of developers.

I'm not wedded to one position or the other but the waterfall on mobile really does suck.

@benmccann
Copy link
Member

Yeah. I don't know if .mjs works for 99% of apps though. It's already been reported broken for nginx, Azure, Go's Fiber server, and a corporate CDN

@dummdidumm can we also sneak in a fix to include mjs on this line? #9141 (comment)

@benmccann
Copy link
Member

Maybe we should see if we can remove the .mjs option and use an ETag alongside the expires header? (guybedford/es-module-shims#354 (comment))

@42triangles
Copy link

42triangles commented Feb 24, 2023

@benmccann I don't think using anything related to headers would actually fix the original issue, considering it uses the static adapter which just outputs a bunch of files, with the headers then only being set by whichever server is serving them.

In my opinion this very much is a breaking change for the static adapter for webservers that do not recognize the .mjs file extension, which in the original issue includes nginx (we're using the Rust library rocket, and it also doesn't set the mimetype correctly, which for our use case (using another type that uses this internally) would require significant changes to support it or an update from rocket - and while rocket is most certainly not nearly as widely used, I do believe that this doesn't change the fact that it is a breaking change due to issues like these).

And as for performance - having an option you can turn on for using .mjs seems reasonable, or at least having the ability to turn it off if the issue is encountered (under the assumption that most bigger servers etc could handle it correctly).

@benmccann
Copy link
Member

To be clear, I was suggesting only serving .js files and removing the .mjs functionality altogether

@42triangles
Copy link

Oh, then I misread that, sorry!

@GeoffreyBooth
Copy link

I'm not wedded to one position or the other but the waterfall on mobile really does suck.

Wasn't this the case until 4-ish days ago? If it was okay until then, I would think it should be okay once again now, at least until a better solution is found. I understand the reluctance to roll back a performance gain, but it's not like the new speed has become so widely used in the last few days that users will react negatively to rolling it back.

There’s nuance here though. Suppose .mjs works for 99% of apps, and 15% of users are on Safari — that means that we’re penalising those 15% of users, not for the sake of 1% of users but 1% of developers.

Unfortunately I don’t think it’s anywhere close to 99% of apps or users. It’s about the servers serving the files, and static webservers are updated on a very long timeline; Apache 2.2 was end-of-life in 2018, but as of 2023-02-23 it’s still used by 3.5% of all websites. Even the current version of Nginx still doesn’t serve .mjs with the correct Content-Type by default, nor does the current version of Apache httpd, even after all these years. Together, Nginx and Apache account for 66.7% of all webservers.

I help manage modules for Node.js, and I’ve been studying this issue for a long time. In 2019 I made a test suite for checking the Content-Type provided for .mjs files by various common webservers: https://github.com/GeoffreyBooth/js-mjs-mime-type-test. Short version: even today, it’s still not safe to assume that .mjs will be served correctly by even the current versions of default-configured static webservers, to say nothing of older versions or CDNs.

I found #9018 (comment), and I must say, I’m very surprised to see that code in Chrome. It feels a lot like a spec violation. As far as I know, browsers aren’t supposed to read “file extensions” (really, the last part of the pathname of a URL) or use them for any purpose, especially not for determining types; that’s what the Content-Type header is for. Either that code is a bug, or maybe an intentional spec violation in the name of performance; but it doesn’t feel like something that should be relied upon. I understand the desire to avoid a waterfall, but I would suggest looking for a less suspect solution than relying on this questionable code.

@@ -1,6 +1,10 @@
/** @type {import('@sveltejs/kit').Config} */
const config = {
kit: {
output: {
preloadStrategy: 'preload-mjs'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we really want to switch our largest test app to a non-default strategy? i'd rather have a separate app test this

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, let's move it into options. the tests need updating anyway by the looks of things

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The recent bug reports around this could have been avoided if your test suite had a test like this:

  1. Build the example app.
  2. Download the latest nginx Docker image and run it, having it serve the build output.
  3. Run headless Chrome and open the port served by the Nginx container.
  4. Verify that there are no errors in the JavaScript console and that the page has minimal expected behavior (like clicking the button increments the counter).

The Docker orchestration parts could be copied from https://github.com/GeoffreyBooth/js-mjs-mime-type-test/blob/master/test.sh.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a huge amount of overhead to guard against a very unlikely regression

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That’s a huge amount of overhead to guard against a very unlikely regression

Completely broken sites also seems pretty bad . . . 🤷

You could write the test and just not include it in your suite, then run it whenever you mess with preload stuff. I would think you’d want a test along these lines, loading the site in a real browser, to check that the waterfall doesn’t happen (in the browsers where you’re not expecting it to).

If you’re currently serving the site like via npm run preview and loading it into headless browsers, you could configure whatever Node-based server npm run preview is using the serve the static files to explicitly serve .mjs files without the Content-Type header (or with a wrong one like application/octet-stream). Then you can avoid the overhead of Docker etc. but still have a test that the site works for the large population of “unable to serve .mjs files” environments.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's simply not going to happen. There are two things we can guard against:

  • re-introducing .mjs, which would require a maintainer to ignore the 'this is why we don't use .mjs' comment that this PR adds
  • every conceivable bug that could happen for every conceivable way someone could use SvelteKit, in which case our test suite would take infinite minutes to run, and we would spend all our lives maintaining the test suite inside of the actual code

Hindsight is 20/20 — it would have been nice not to have introduced this breaking change, but the reality of software development is that breakage happens. That's why we have versions!

link_header_preloads.add(`<${encodeURI(path)}>; rel="modulepreload"; nopush`);
head += `\n\t\t<link rel="preload" as="script" crossorigin="anonymous" href="${path}">`;
if (options.preload_strategy !== 'modulepreload') {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm, is the rel=modulepreload link always added ? shouldn't it be either one or the other?

Copy link
Member Author

@dummdidumm dummdidumm Feb 24, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The modulepreload equivalent is the header links above. Since it's a header it doesn't hurt I think (and it always was like this), but you're right we could put it inside an if block

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Chrome doesn't respect preload headers, only the <link>. This might be a bug in Chrome, but I haven't got round to investigating and/or filing a ticket. Either way, I think we need to keep the modulepreload link headers for Chrome's benefit

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wait, really? This means that chrome didn't get preload behavior previously, too, because that logic I implemented is basically the one we had prior to your PR which introduced the link preload. Mhmmm.
Isn't there also a related issue about the link headers being too large and crashing something? Do we need another "linkheaders" option?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure I follow? we previously added modulepreload links to the HTTP header and still do (and should)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as for the 'link headers being too large' thing, that's why we introduced the preload option to resolve

* What preload strategy to use for JavaScript files to avoid waterfalls:
* - `modulepreload` (default) - uses `<link rel="modulepreload">` to preload JavaScript files. Works only in Chromium-based browsers currently, and soon in Safari, too.
* - `preload-js` - uses `<link rel="preload">` to preload JavaScript files. Works in all browsers but Firefox, and causes double-parsing of the script for Chromium-based browser.
* - `preload-mjs` - uses `<link rel="preload">` to preload JavaScript files, but uses the `.mjs` extension. Works in Chromium-based browsers and Safari. Check of your provider/CDN has the correct MIME type for `.mjs` files before turning this on.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* - `preload-mjs` - uses `<link rel="preload">` to preload JavaScript files, but uses the `.mjs` extension. Works in Chromium-based browsers and Safari. Check of your provider/CDN has the correct MIME type for `.mjs` files before turning this on.
* - `preload-mjs` - uses `<link rel="preload">` to preload JavaScript files, but uses the `.mjs` extension. Works in Chromium-based browsers and Safari. Check if your server or CDN sets the correct `Content-Type` header for `.mjs` files (text/javascript) before turning this on; browsers will error on a missing or incorrect `Content-Type` header.

Thank you, this overall comment is much clearer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Project builds but fails on deploy
7 participants