unhandled error with browser tests and modified base

[alexdulin]

Describe the bug

Browser testing does not work when using a non-default base setting. When the base value is modified, the following error occurs using a minimal vitest.config.ts

import { defineConfig } from 'vitest/config';

export default defineConfig({
  base: '/example/',
  test: {
    browser: {
      enabled: true,
      headless: true,
      name: 'firefox',
      provider: 'playwright',
    }
  }
})

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯ Unhandled Errors ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯

Vitest caught 1 unhandled error during the test run.
This might cause false positive tests. Resolve unhandled errors to make sure your tests are not affected.

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯ Reload Error ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
Error: Vitest failed to load "vitest/utils" after 10 retries.
 ❯ ../../../../../__vitest_browser__/index-JmUxqpfn.js:958:36
 ❯ async* ../../../../../__vitest_browser__/index-JmUxqpfn.js:950:4

Caused by: TypeError: error loading dynamically imported module: http://localhost:5173/@id/vitest/utils

Removing the base setting, or setting it to an empty string, makes the error no longer occur.

Reproduction

This minimal example demonstrates the issue. Run npm install; npm test to cause the error. Remove the base configuration option in vitest.config.ts to make the error go away.

vitest-browser-base-bug.zip

System Info

System:
    OS: Linux 6.2 Ubuntu 22.04.3 LTS 22.04.3 LTS (Jammy Jellyfish)
    CPU: (16) x64 Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz
    Memory: 56.16 GB / 62.63 GB
    Container: Yes
    Shell: 5.1.16 - /bin/bash
  Binaries:
    Node: 20.5.1 - /usr/bin/node
    Yarn: 1.22.19 - /usr/bin/yarn
    npm: 10.2.2 - ~/.npm-packages/bin/npm
  Browsers:
    Chrome: 120.0.6099.62
  npmPackages:
    @vitest/browser: ^1.0.1 => 1.0.1 
    vite: ^5.0.6 => 5.0.6 
    vitest: ^1.0.1 => 1.0.1

Used Package Manager

npm

Validations

• Follow our Code of Conduct
• Read the Contributing Guidelines.
• Read the docs.
• Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
• Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
• The provided reproduction is a minimal reproducible example of the bug.

[mbland]

@alexdulin Beat me to this today...I've got the root cause:

• fix!: return 404 for resources requests outside the base path vitejs/vite#5657 made Vite reject URLs outside of config.base.
• This change became part of Vite 5.0.0-beta.8
• Vitest's dependency on Vite was bumped to ^5.0.0-beta.15 as part of chore!: update vite@5 and rollup@4 #4368
• This change became part of Vitest 1.0.0-beta.5

I've just posted the repo with my minimal demo and notes on how to reproduce these findings: https://github.com/mbland/vitest-utils-browser-import

And FWIW, I went through the steps outlined in the above repo to pinpoint this after reproducing the problem in one of my other projects on:

• macOS: Safari, Chrome, Firefox
• Linux: Chromium, Firefox
• Windows 11: Chrome, Firefox, Edge
• GitHub Actions + Linux: Chrome

I'm happy to try to take a stab at a PR myself in the next day or two, if I'm able to figure out a good way to fix it before anyone else does.

[sheremet-va]

I'm happy to try to take a stab at a PR myself in the next day or two, if I'm able to figure out a good way to fix it before anyone else does.

I think we just need to pass down base to importId:

vitest/packages/browser/src/client/utils.ts

Line 1 in 640881a

export async function importId(id: string) {

[mbland]

I think we just need to pass down base to importId:

vitest/packages/browser/src/client/utils.ts

Line 1 in 640881a

export async function importId(id: string) {

Cool. I'd traced through the code and wondered if passing base there was the right approach, so I'll give that a try.

[hi-ogawa]

I wonder if using base config is genuinely required to run Vitest in your current setup.
If base config is simply coming from the shared vite.config.ts used for main Vite app and disabling it during Vitest is acceptable, then using process.env.VITEST to change this option might be a quick workaround for the time being, for example:

export default defineConfig({
  base: process.env.VITEST ? undefined : '/example/',
  test: {
    browser: {
      enabled: true,
      headless: true,
      name: 'firefox',
      provider: 'playwright',
    }
  }
})

[alexdulin]

@hi-ogawa That is exactly the case (where the base setting is coming from a shared vite.config.ts) and the workaround we have done in the mean time. However, it would still be nice not to have to add any extra configuration and complexity to work around this. Until a proper fix is in though, a solution like what you suggest certainly does work.

[mbland]

FWIW, I've also updated the test in my demo showing how the workaround will timeout when opening a local page prefixed with config.base: https://github.com/mbland/vitest-utils-browser-import/blob/1855ea285d7b88117b91c7d1b6d5b819eb271af5/demo.test.js#L1-L11