Fix running server with Polyfilled fetch

[javivelasco]

This PR fixes #30398

By default Next will polyfill some fetch APIs (Request, Response, Header and fetch) only if fetch is not found in the global scope in certain entry points. If we have a custom server which is adding a global fetch (and only fetch) at the very top then the rest of APIs will not be polyfilled.

This PR adds a test on the custom server where we can add a custom polyfill for fetch with an env variable. This reproduces the issue since next-server.js will be required without having a polyfill for Response which makes it fail on requiring NextResponse. Then we remove the code that checks for subrequests to happen within the sandbox so that we don't need to polyfill next-server anymore.

The we also introduce an improvement on how we handle relative requests. Since #31858 introduced a port and hostname options for the server, we can always pass absolute URLs to the Middleware so we can always use the original nextUrl to pass it to fetch. This brings a lot of simplification for NextURL since we don't have to consider relative URLs no more.

Bug

• Related issues linked using fixes #number
• Integration tests added
• Errors have helpful link attached, see contributing.md

[ijjk]

Failing test suites

Commit: ec74432

test/production/middleware-typescript/test/index.test.ts

• should set-up next > should have built and started

Expand output

● should set-up next › should have built and started

thrown: "Exceeded timeout of 90000 ms for a hook.
Use jest.setTimeout(newTimeout) to increase the timeout value, if this is a long-running test."

   6 | import { renderViaHTTP } from 'next-test-utils'
   7 |
>  8 | const appDir = join(__dirname, '../app')
     |     ^
   9 |
  10 | describe('should set-up next', () => {
  11 |   let next: NextInstance

  at production/middleware-typescript/test/index.test.ts:8:5
  at Object.<anonymous> (production/middleware-typescript/test/index.test.ts:6:1)

● Test suite failed to run

TypeError: Cannot read property 'destroy' of undefined

  20 |       dependencies: {
  21 |         typescript: 'latest',
> 22 |         '@types/react': 'latest',
     |                       ^
  23 |         '@types/react-dom': 'latest',
  24 |       },
  25 |     })

  at production/middleware-typescript/test/index.test.ts:22:23

[ijjk]

Stats from current PR

Default Build (Increase detected ⚠️)

General Overall increase ⚠️

	vercel/next.js canary	javivelasco/next.js fix-30398	Change
buildDuration	23.8s	23.2s	-603ms
buildDurationCached	4.2s	4.2s	-23ms
nodeModulesSize	347 MB	347 MB	⚠️ +5.36 kB

Page Load Tests Overall increase ✓

	vercel/next.js canary	javivelasco/next.js fix-30398	Change
/ failed reqs	0	0	✓
/ total time (seconds)	3.965	3.999	⚠️ +0.03
/ avg req/sec	630.55	625.21	⚠️ -5.34
/error-in-render failed reqs	0	0	✓
/error-in-render total time (seconds)	2.057	2.004	-0.05
/error-in-render avg req/sec	1215.64	1247.6	+31.96

Client Bundles (main, webpack, commons)

	vercel/next.js canary	javivelasco/next.js fix-30398	Change
450.HASH.js gzip	179 B	179 B	✓
framework-HASH.js gzip	42.2 kB	42.2 kB	✓
main-HASH.js gzip	28.4 kB	28.4 kB	✓
webpack-HASH.js gzip	1.45 kB	1.45 kB	✓
Overall change	72.3 kB	72.3 kB	✓

Legacy Client Bundles (polyfills)

	vercel/next.js canary	javivelasco/next.js fix-30398	Change
polyfills-HASH.js gzip	31 kB	31 kB	✓
Overall change	31 kB	31 kB	✓

Client Pages

	vercel/next.js canary	javivelasco/next.js fix-30398	Change
_app-HASH.js gzip	1.37 kB	1.37 kB	✓
_error-HASH.js gzip	194 B	194 B	✓
amp-HASH.js gzip	312 B	312 B	✓
css-HASH.js gzip	326 B	326 B	✓
dynamic-HASH.js gzip	2.38 kB	2.38 kB	✓
head-HASH.js gzip	350 B	350 B	✓
hooks-HASH.js gzip	635 B	635 B	✓
image-HASH.js gzip	4.49 kB	4.49 kB	✓
index-HASH.js gzip	263 B	263 B	✓
link-HASH.js gzip	1.87 kB	1.87 kB	✓
routerDirect..HASH.js gzip	321 B	321 B	✓
script-HASH.js gzip	383 B	383 B	✓
withRouter-HASH.js gzip	318 B	318 B	✓
85e02e95b279..7e3.css gzip	107 B	107 B	✓
Overall change	13.3 kB	13.3 kB	✓

Client Build Manifests

	vercel/next.js canary	javivelasco/next.js fix-30398	Change
_buildManifest.js gzip	460 B	460 B	✓
Overall change	460 B	460 B	✓

Rendered Page Sizes

	vercel/next.js canary	javivelasco/next.js fix-30398	Change
index.html gzip	532 B	532 B	✓
link.html gzip	545 B	545 B	✓
withRouter.html gzip	526 B	526 B	✓
Overall change	1.6 kB	1.6 kB	✓

---

Default Build with SWC (Decrease detected ✓)

General Overall increase ⚠️

	vercel/next.js canary	javivelasco/next.js fix-30398	Change
buildDuration	26.5s	26.3s	-161ms
buildDurationCached	4.3s	4.2s	-137ms
nodeModulesSize	347 MB	347 MB	⚠️ +5.36 kB

Page Load Tests Overall decrease ⚠️

	vercel/next.js canary	javivelasco/next.js fix-30398	Change
/ failed reqs	0	0	✓
/ total time (seconds)	3.876	3.977	⚠️ +0.1
/ avg req/sec	645.02	628.62	⚠️ -16.4
/error-in-render failed reqs	0	0	✓
/error-in-render total time (seconds)	2.032	2.03	0
/error-in-render avg req/sec	1230.15	1231.31	+1.16

Client Bundles (main, webpack, commons)

	vercel/next.js canary	javivelasco/next.js fix-30398	Change
450.HASH.js gzip	179 B	179 B	✓
framework-HASH.js gzip	42.3 kB	42.3 kB	✓
main-HASH.js gzip	28.6 kB	28.6 kB	✓
webpack-HASH.js gzip	1.44 kB	1.44 kB	✓
Overall change	72.5 kB	72.5 kB	✓

Legacy Client Bundles (polyfills)

	vercel/next.js canary	javivelasco/next.js fix-30398	Change
polyfills-HASH.js gzip	31 kB	31 kB	✓
Overall change	31 kB	31 kB	✓

Client Pages

	vercel/next.js canary	javivelasco/next.js fix-30398	Change
_app-HASH.js gzip	1.35 kB	1.35 kB	✓
_error-HASH.js gzip	180 B	180 B	✓
amp-HASH.js gzip	305 B	305 B	✓
css-HASH.js gzip	321 B	321 B	✓
dynamic-HASH.js gzip	2.37 kB	2.37 kB	✓
head-HASH.js gzip	342 B	342 B	✓
hooks-HASH.js gzip	622 B	622 B	✓
image-HASH.js gzip	4.53 kB	4.53 kB	✓
index-HASH.js gzip	256 B	256 B	✓
link-HASH.js gzip	1.9 kB	1.9 kB	✓
routerDirect..HASH.js gzip	314 B	314 B	✓
script-HASH.js gzip	375 B	375 B	✓
withRouter-HASH.js gzip	309 B	309 B	✓
85e02e95b279..7e3.css gzip	107 B	107 B	✓
Overall change	13.3 kB	13.3 kB	✓

Client Build Manifests

	vercel/next.js canary	javivelasco/next.js fix-30398	Change
_buildManifest.js gzip	458 B	458 B	✓
Overall change	458 B	458 B	✓

Rendered Page Sizes

	vercel/next.js canary	javivelasco/next.js fix-30398	Change
index.html gzip	532 B	532 B	✓
link.html gzip	545 B	545 B	✓
withRouter.html gzip	526 B	526 B	✓
Overall change	1.6 kB	1.6 kB	✓

Commit: d027c2b