Improve throughput of stale image requests

[styfle]

I discovered that serving stale images wasn't any faster because revalidation wasn't happening in the background, it was actually blocking concurrent requests due to inflightRequests promise. So this PR resolves with the stale buffer so that any concurrent requests during the revalidation period can still serve the stale image right away.

• Follow up to Add stale-while-revalidate pattern to Image Optimization API #33735
• Follow up to Fix image cache race condition #33883
• Related to "ENOENT: no such file or directory" - next.js crashes, related to image caching? #33860
• Related to Image Optimization should use the stale-while-revalidate pattern #27208

[ijjk]

Stats from current PR

Default Build (Increase detected ⚠️)

General Overall increase ⚠️

	vercel/next.js canary	vercel/next.js stale-request-concurrency-in-memory-cache	Change
buildDuration	18.5s	18.5s	⚠️ +17ms
buildDurationCached	7.2s	7.1s	-113ms
nodeModulesSize	359 MB	359 MB	⚠️ +3.76 kB

Page Load Tests Overall increase ✓

	vercel/next.js canary	vercel/next.js stale-request-concurrency-in-memory-cache	Change
/ failed reqs	0	0	✓
/ total time (seconds)	3.776	3.807	⚠️ +0.03
/ avg req/sec	662.04	656.68	⚠️ -5.36
/error-in-render failed reqs	0	0	✓
/error-in-render total time (seconds)	1.613	1.583	-0.03
/error-in-render avg req/sec	1549.99	1579.65	+29.66

Client Bundles (main, webpack, commons)

	vercel/next.js canary	vercel/next.js stale-request-concurrency-in-memory-cache	Change
450.HASH.js gzip	179 B	179 B	✓
framework-HASH.js gzip	42 kB	42 kB	✓
main-HASH.js gzip	27.4 kB	27.4 kB	✓
webpack-HASH.js gzip	1.44 kB	1.44 kB	✓
Overall change	71.1 kB	71.1 kB	✓

Legacy Client Bundles (polyfills)

	vercel/next.js canary	vercel/next.js stale-request-concurrency-in-memory-cache	Change
polyfills-HASH.js gzip	31 kB	31 kB	✓
Overall change	31 kB	31 kB	✓

Client Pages

	vercel/next.js canary	vercel/next.js stale-request-concurrency-in-memory-cache	Change
_app-HASH.js gzip	1.36 kB	1.36 kB	✓
_error-HASH.js gzip	194 B	194 B	✓
amp-HASH.js gzip	312 B	312 B	✓
css-HASH.js gzip	326 B	326 B	✓
dynamic-HASH.js gzip	2.57 kB	2.57 kB	✓
head-HASH.js gzip	350 B	350 B	✓
hooks-HASH.js gzip	919 B	919 B	✓
image-HASH.js gzip	5.01 kB	5.01 kB	✓
index-HASH.js gzip	263 B	263 B	✓
link-HASH.js gzip	2.26 kB	2.26 kB	✓
routerDirect..HASH.js gzip	321 B	321 B	✓
script-HASH.js gzip	383 B	383 B	✓
withRouter-HASH.js gzip	318 B	318 B	✓
85e02e95b279..7e3.css gzip	107 B	107 B	✓
Overall change	14.7 kB	14.7 kB	✓

Client Build Manifests

	vercel/next.js canary	vercel/next.js stale-request-concurrency-in-memory-cache	Change
_buildManifest.js gzip	458 B	458 B	✓
Overall change	458 B	458 B	✓

Rendered Page Sizes

	vercel/next.js canary	vercel/next.js stale-request-concurrency-in-memory-cache	Change
index.html gzip	532 B	532 B	✓
link.html gzip	544 B	544 B	✓
withRouter.html gzip	526 B	526 B	✓
Overall change	1.6 kB	1.6 kB	✓

---

Default Build with SWC (Decrease detected ✓)

General Overall increase ⚠️

	vercel/next.js canary	vercel/next.js stale-request-concurrency-in-memory-cache	Change
buildDuration	22.3s	22.5s	⚠️ +175ms
buildDurationCached	7.1s	7.2s	⚠️ +74ms
nodeModulesSize	359 MB	359 MB	⚠️ +3.76 kB

Page Load Tests Overall decrease ⚠️

	vercel/next.js canary	vercel/next.js stale-request-concurrency-in-memory-cache	Change
/ failed reqs	0	0	✓
/ total time (seconds)	3.74	3.798	⚠️ +0.06
/ avg req/sec	668.41	658.18	⚠️ -10.23
/error-in-render failed reqs	0	0	✓
/error-in-render total time (seconds)	1.593	1.613	⚠️ +0.02
/error-in-render avg req/sec	1569.48	1550.01	⚠️ -19.47

Client Bundles (main, webpack, commons)

	vercel/next.js canary	vercel/next.js stale-request-concurrency-in-memory-cache	Change
450.HASH.js gzip	179 B	179 B	✓
framework-HASH.js gzip	42.1 kB	42.1 kB	✓
main-HASH.js gzip	27.4 kB	27.4 kB	✓
webpack-HASH.js gzip	1.44 kB	1.44 kB	✓
Overall change	71.2 kB	71.2 kB	✓

Legacy Client Bundles (polyfills)

	vercel/next.js canary	vercel/next.js stale-request-concurrency-in-memory-cache	Change
polyfills-HASH.js gzip	31 kB	31 kB	✓
Overall change	31 kB	31 kB	✓

Client Pages

	vercel/next.js canary	vercel/next.js stale-request-concurrency-in-memory-cache	Change
_app-HASH.js gzip	1.35 kB	1.35 kB	✓
_error-HASH.js gzip	180 B	180 B	✓
amp-HASH.js gzip	305 B	305 B	✓
css-HASH.js gzip	321 B	321 B	✓
dynamic-HASH.js gzip	2.56 kB	2.56 kB	✓
head-HASH.js gzip	342 B	342 B	✓
hooks-HASH.js gzip	911 B	911 B	✓
image-HASH.js gzip	5.05 kB	5.05 kB	✓
index-HASH.js gzip	256 B	256 B	✓
link-HASH.js gzip	2.28 kB	2.28 kB	✓
routerDirect..HASH.js gzip	314 B	314 B	✓
script-HASH.js gzip	375 B	375 B	✓
withRouter-HASH.js gzip	309 B	309 B	✓
85e02e95b279..7e3.css gzip	107 B	107 B	✓
Overall change	14.7 kB	14.7 kB	✓

Client Build Manifests

	vercel/next.js canary	vercel/next.js stale-request-concurrency-in-memory-cache	Change
_buildManifest.js gzip	460 B	460 B	✓
Overall change	460 B	460 B	✓

Rendered Page Sizes

	vercel/next.js canary	vercel/next.js stale-request-concurrency-in-memory-cache	Change
index.html gzip	534 B	534 B	✓
link.html gzip	548 B	548 B	✓
withRouter.html gzip	528 B	528 B	✓
Overall change	1.61 kB	1.61 kB	✓

Commit: 7d793ce