Cache-Control headers set in next.config.js are overwritten

[hartshorne]

What version of Next.js are you using?

10.0.7, 10.0.8-canary.3

What version of Node.js are you using?

15.8.0

What browser are you using?

curl, Chrome

What operating system are you using?

macOS

How are you deploying your application?

next start

Describe the Bug

Custom Cache-Control headers configured in next.config.js are overwritten in some cases. It looks any page that use getStaticProps will have their cache headers overwritten with Cache-Control: s-maxage=31536000, stale-while-revalidate which seems to come from

next.js/packages/next/next-server/server/send-payload.ts

Line 34 in 80c9522

res.setHeader('Cache-Control', `s-maxage=31536000, stale-while-revalidate`)

Expected Behavior

When we configure a Cache-Control header, don't set it to something else.

To Reproduce

Starting from yarn create next-app --example headers headers-app, here's a sample project that demonstrates the bug:
https://github.com/hartshorne/headers-app

You can clone the project, and run something like yarn build && yarn start to start it in production mode. (dev mode overwrites all Cache-Control headers to prevent the browser from caching during development, which makes sense.)

Here's the next.config.js: https://github.com/hartshorne/headers-app/blob/main/next.config.js

Here's props.js (which exports getStaticProps) – this is broken: https://github.com/hartshorne/headers-app/blob/main/pages/props.js

% curl -I http://localhost:3000/props 
HTTP/1.1 200 OK
Cache-Control: s-maxage=31536000, stale-while-revalidate
X-Custom-Header: with static props
X-Powered-By: Next.js
ETag: "978-AzRqkosC2YfRQvbfuY7KiKb51e8"
Content-Type: text/html; charset=utf-8
Content-Length: 2424
Vary: Accept-Encoding
Date: Thu, 18 Feb 2021 22:09:41 GMT
Connection: keep-alive
Keep-Alive: timeout=5

Here's static.js — this works: https://github.com/hartshorne/headers-app/blob/main/pages/props.js

% curl -I http://localhost:3000/static     
HTTP/1.1 200 OK
Cache-Control: public, max-age=3600, s-maxage=60, stale-while-revalidate=86400
X-Custom-Header: no props, no link
X-Powered-By: Next.js
ETag: "945-MX0a4VCO/O001kyDcNn9a55taLQ"
Content-Type: text/html; charset=utf-8
Content-Length: 2373
Vary: Accept-Encoding
Date: Thu, 18 Feb 2021 22:09:10 GMT
Connection: keep-alive
Keep-Alive: timeout=5

Note that X-Custom-Header comes through, but the Cache-Control header is overwritten. Same behavior in Chrome, and with a regular GET request.

[hartshorne]

@ijjk, it looks like a32b1f4 introduced code that would overwrite custom Cache-Control headers -- maybe we can check to see if the Cache-Control headers were set before overwriting them?

[hartshorne]

The bug exists in https://github.com/vercel/next.js/releases/tag/v10.0.8-canary.3 as well: https://github.com/hartshorne/headers-app/tree/canary

[hartshorne]

Related: #19914

[hartshorne]

This would fix the issue:
canary...hartshorne:preserve-custom-cache-control-headers

But seems to run against the grain of the documentation:
https://nextjs.org/docs/api-reference/next.config.js/headers#cache-control

Cache-Control headers set in next.config.js will be overwritten in production to ensure that static assets can be cached effectively. 

@timneutkens reasonable defaults are great, but it seems like there should be a way to easily override something like a Cache-Control header.

[FDiskas]

it works for me on [email protected] - all images served from public directory with cache control header at next.config.js

  async headers() {
    return [
      {
        source: '/:all*(svg|jpg|png)',
        locale: false,
        headers: [
          {
            key: 'Cache-Control',
            value: 'public, max-age=9999999999, must-revalidate',
          }
        ],
      },
    ]
  },

[ramiel]

Any news on this? I have an api endpoint that produces images and I want to cache them on the client. Why can't I specify the desired cache-control headers?

When the documentation says

Cache-Control headers set in next.config.js will be overwritten in production to ensure that static assets can be cached effectively.

makes too many assumptions on the user use-case.

[FDiskas]

@ramiel vote for #23328 :)

[ramiel]

@FDiskas I voted for that one, but it's not exactly the same issue. That is specific for the images through next/image I guess, this is for anything

[FDiskas]

Take a look at my example in the comments, this works on production as well

[flockonus]

This is a considerable unchangeable default can cause pages to be stale up to 1 years for CDNs, which is bonkers(!)
And agree with @ramiel , these are too many assumptions about use case, for a config that can't be changed.

Also note that stale-while-revalidate seems to be used being used incorrectly, as it should have =seconds to it

[schlosser]

Big +1 here – when using next/image with images from the /public folder deployed outside of Vercel, we're getting:

cache-control: public, max-age=0, must-revalidate

No matter what we put in next.config.js. This makes our images load extremely slow. We'd like our CDN to use stale-while-revalidate behavior but we can't get our webserver to respect this setting...

[cweekly]

I'm flabbergasted by:

"Cache-Control headers set in next.config.js will be overwritten in production to ensure that static assets can be cached effectively."

Sane defaults would be one thing. But this is just nuts.
"Using next.js means you cannot configure your app's HTTP response headers."

Please, maintainers, provide some way to opt out of this heavy-handed, unpredictable, one-size-fits-some behavior.
Please.

[yarolegovich]

Was quite disappointed to find this out.
The plan was to have the following setup for a hosted CMS:

• Content creators have their pages that change infrequently.
• When a page is requested incremental static generation is used and the page is cached.
• When a page is updated path is revalidated.
  The problem is revalidate_unstable can be used to invalidate the file on the server, but it's still stuck in a CDN.

I'm using CDN-Cache-Control header to overwrite CDN s-maxage, but it's frustrating we can't control resources headers 👎

[jekh]

Is there truly no way to set your own cache-control headers for statically-generated pages? You can set the s-maxage value by remembering to set revalidate in every return statement within every getStaticProps(), but what if you don't want to use the default s-maxage=X, stale-while-revalidate pattern for the cache-control header?

Would a middleware potentially work for getStaticProps pages? Does that fire for all requests for static pages, even when the page is still fresh? If so, will setting response headers still work with the new middleware?

The default for getStaticProps pages seems like quite the footgun: unlike every other page, it automatically adds a very high default s-maxage -- don't forget, or that page could be stuck in your CDN forever!

[imconfused218]

Really frustrating. Seeing bugs with ISR pages that will sometimes default to s-maxage=31536000 instead of using their revalidate page setting. Pages that should be re-generating every 2 minutes are suddenly stuck for a year. Please consider setting more appropriate defaults or allow a cache-control or max-age configuration

[yangxuHBO]

async headers() {
    return [
      {
        source: '/:all*(svg|jpg|png)',
        locale: false,
        headers: [
          {
            key: 'Cache-Control',
            value: 'public, max-age=9999999999, must-revalidate',
          }
        ],
      },
    ]
  },

My nextJS is v11. Did not work for me. Still got some other max-age rather than this one.

[Mihai-github]

Hi, I'm having the same issue with a nex.js website with a static build hosted on S3.
Even though I'm setting in the next.config.js file rule for cache-control:

async headers() {
    return [
      {
        source: '/:all*(svg|jpg|png)',
        locale: false,
        headers: [
          {
            key: 'Cache-Control',
            value: 'public, max-age=9999999999, must-revalidate',
          }
        ],
      },
    ]
  },

In network tab I end up with Request Headers cache-control: no-cache

[AdamZaczek]

We've spent a week trying to fix this issue. The app is broken for thousands of CodeAlly users because of this behavior.
We use Vercel + Next and migrated from Gatsby.
Right now, we can not serve a production fix for our user base because cache is stuck.

We need an option to set the cache header or we can't deliver the product in any shape or form. Please.

[iddk0321]

I deployed my app using the standalone option to create a server.js file, and then deployed it using Docker. I'm not sure if this is the best method, but it works as I want, so I'm sharing it.

When building with the Dockerfile, add the following command:

RUN sed -i 's/public, max-age=31536000, immutable/no-cache, no-store, must-revalidate/' ./node_modules/next/dist/server/next-server.js

The 'sed' command is similar to JavaScript's replace, which finds the text you want and replaces it with another. In the node_modules/next/dist/server/next-server.js file, you can see that the Cache-Control header is overridden as 'public, max-age=31536000, immutable'. (Search for the setImmutableAssetCacheControl function.)

You can change this code to the cache mode you want. I changed it to use the 'no-cache, no-store, must-revalidate' option.

Below is my complete Dockerfile.

FROM node:18-alpine AS base

FROM base AS deps

RUN apk add --no-cache libc6-compat
WORKDIR /app

COPY package.json yarn.lock* package-lock.json* pnpm-lock.yaml* ./
RUN \
  if [ -f yarn.lock ]; then yarn --frozen-lockfile; \
  elif [ -f package-lock.json ]; then npm ci; \
  elif [ -f pnpm-lock.yaml ]; then yarn global add pnpm && pnpm i --frozen-lockfile; \
  else echo "Lockfile not found." && exit 1; \
  fi

FROM base AS builder
WORKDIR /app

COPY --from=deps /app/node_modules ./node_modules

COPY . .

RUN yarn build

FROM base AS runner
WORKDIR /app

ENV NODE_ENV production
RUN addgroup --system --gid 1001 nodejs
RUN adduser --system --uid 1001 nextjs

COPY --from=builder /app/public ./public

RUN mkdir .next
RUN chown nextjs:nodejs .next
RUN yarn global add pm2
RUN npm install sharp

COPY ecosystem.config.js ./ecosystem.config.js
COPY --from=builder --chown=nextjs:nodejs /app/.next/standalone ./
COPY --from=builder --chown=nextjs:nodejs /app/.next/static ./.next/static

USER nextjs

EXPOSE 9300
ENV PORT 9300
ENV HOSTNAME "0.0.0.0"

RUN sed -i 's/public, max-age=31536000, immutable/no-cache, no-store, must-revalidate/' ./node_modules/next/dist/server/next-server.js

CMD ["pm2-runtime", "start", "ecosystem.config.js"]

[robhrt7]

RE: @joekur

@leerob how is this supposed to work when you want to deploy a new version of your application? I don't know if vercel is doing something special, but if you're self hosting, and you have a CDN in front of your app, how are you expected to be able to ever make updates?

My best guess after looking at vercel's architecture is that the cache is somehow specific to a particular deploy, whereas when self hosting the cache will simply be based on the requested URL?

This is a major issue for our team that self hosts next.js with a cloudfront distribution.

It does look that Next.js cache headers approach is mainly implemented based on what is true on Vercel platform, as is biased in some way or another. It's indeed an important point that Vercel does invalidate cache much better across deploys, being a highly tailored hosting solution for Next.js, while other CDN services are much more generic purpose and in some cases implementation specifics get in a way of what Next.js output as default.

As different CDN vendors implement their infra differently, and many features that present in the HTTP spec, are either implemented differently across the board, or not supported at all, it's very hard to navigate this topic for different circumstances.

I do wish that Next.js power users could have an ability to override cache headers, and I'm continuously surprised how is this not addressed over the years people been facing these issue. I guess the % of people who care about efficient caching on self-hosted Next.js is still too small and it's easy to get lost in the topic. But with more enterprise adoption, I'm sure Next.js will be forced to give more controls, as I don't see any reason not doing this.

I'm currently working on some articles to cover some Next.js self host issues, to be covered in FocusReactive blog. Hoping to get to the bottom of this at some point.

Chipping in on the solutions options on how things stand right now (what I'll also cover in articles in more detail) - you can partially override some cache headers, for example for fully static pages without data fetching, despite what Next.js docs state. And the option we chose is to add CDN specific cache control headers that are respected by Fastly over the default cache control - Surrogate-Control (note that it's a proprietary header for Fastly)

[joekur]

Not ideal, but we ended up working around this limitation by using a custom server and monkey-patching the http writeHead method. This method seemed slightly less fragile compared to overwriting next's source more directly. The constraints were that Next would overwrite any Cache-Control header already set, and by the time you call handleNextRequest(), the response will already have been written back to the request. So before handing off to handleNextRequest():

  const originalWriteHead = res.writeHead;

  res.writeHead = (...args) => {
    res.setHeader(
      'Cache-Control',
      'private, no-cache, no-store, max-age=0, must-revalidate',
    );

    return originalWriteHead.apply(res, args);
  }

Of course if Next ever changes their implementation to not call writeHead, this will break 😞

This solution fully turns off CDN and local caching for all HTML requests, which for us 1. ensures that middleware will always get run, 2. removes some of the undesirable caching behavior when you mix ISR with a CDN cache, and 3. fixes the issue where static routes were getting cached for a full year causing newly deployed changes to never be seen.

[calvinl]

running across this issue while looking for a way to cache with origin headers in Google's Cloud CDN, which requires 'Cache-Control' to be set with maxage. Really disappointing this hasn't been addressed.

[ConnorLanglois]

Just coming across this bug myself - this seems like a major issue in such a "high profile" framework like NextJS. Honestly, it is very concerning that this wasn't addressed years ago, as this affects anybody self-hosting with truly static pages (i.e. not using revalidate). Anytime you push a new update to your server, the changes won't actually take place since the cache-control headers have caused intermediary cdn's (and proxies!!) to cache the prior version. Major L.

Of course, one can just purge their cdn after updating their server (and hopefully your cdn hasn't been forwarding your cache-control headers...), but what if one doesn't have a cdn sitting in front of their server? Then anybody accessing your site through a proxy will receive old pages for a whole entire year (since s-maxage applies to any shared caches, and a proxy is a shared cache). Wild that this issue actually exists in this framework, I am bewildered that this is the default behavior, and that there is no way to opt-out of it.

[dkpoult]

Same issue as others, we noticed our GCP Cloud Run + CDN site was breaking after we deployed changes because the HTML had references to code that was no longer available. Eventually realised that CDN was caching our statically generated home page (for up to a year??) and that when we rebuild the site on deploy it would cause some file hashes to be change and 404s from the stale HTML trying to fetch them anyways.

Our solution so far has been an entirely unnecessary switch from SSG to SSR because there's just no practical way to override the cache header without us migrating our infrastructure. This is honestly such a mind boggling decision by the Vercel team, and at this point the super opinionated roadblocks that I've hit in adopting NextJS have been enough for me to start recommending against it to other devs considering it.

[lhguerra]

@joekur where did you apply this code? Are you using app or pages router?

[joekur]

@joekur where did you apply this code? Are you using app or pages router?

@lhguerra in a custom server, before handing off the request to the app.getRequestHandler().

[lhguerra]

Hm that's what I was afraid of, this won't work in app router :/

[flyjeray]

Thank you @u3u! It appears, our problem is not fixed even with your version of the revalidate0headers file but this might help others.

@AdamZaczek Answering a year later, but maybe it will be useful for you or the others :)

You might have (or might have not) been confused, but Cache-Control is also overridden for assets (CSS, JS, Fonts, Images..) on the dev server, which will ignore the fix by @u3u. Noting it down here for no one to spend their time on looking for a fix.

It can be fixed by extending patch to include development config:

--- a/node_modules/next/dist/server/dev/next-dev-server.js
+++ b/node_modules/next/dist/server/dev/next-dev-server.js
@@ -832,7 +832,9 @@ class DevServer extends _nextServer.default {
         return await (0, _loadComponents).loadDefaultErrorComponents(this.distDir);
     }
     setImmutableAssetCacheControl(res) {
-        res.setHeader("Cache-Control", "no-store, must-revalidate");
+        if (!res.getHeader("Cache-Control")) {
+            res.setHeader("Cache-Control", "no-store, must-revalidate");
+        }
     }

While the patch above will fix the issues in production, this one will do the thing during development

Note: version of NextJS used is 12.2.5

[eloisetaylor5693]

Still experiencing this issue in v14
#62294

[ccchapman]

We are finding this a major hurdle that we cannot control the Cache-Control header from our Next.js SSR application.

[6stringbeliever]

Allow me to add to the growing chorus of people affected by this issue. Looking to implement a solution similar to @joekur above, but truly disheartening to see how many people are facing this issue and the lack of response from Vercel, particularly with an active pull request.

[ccchapman]

Below is a workaround which will let you set a header from Next.js middleware and then rewrite the Cache-Control header from nginx configuration.

function cacheControlMiddleware(): NextResponse | undefined {
  const response = NextResponse.next();
  response.headers.set(
    "X-Cache-Control-Override",
    "public, max-age=60, s-maxage=60, stale-while-revalidate=3600, stale-if-error=86400",
  );
  return response;
}

http {
    map $upstream_http_x_cache_control_override $cache_control_override {
        default $upstream_http_cache_control;
        "~.+?" $upstream_http_x_cache_control_override;
    }
    
    server {
      # ...
        
      location / {
          proxy_pass http://127.0.0.1:3000;
          # ...
          proxy_hide_header Cache-Control;
          proxy_hide_header X-Cache-Control-Override;
          add_header Cache-Control $cache_control_override;
      }
}

[lhguerra]

How would we set this custom Nginx configuration at Vercel?

[cvoege]

What about an optional field in the generateMetadata response for cacheControl? Or more generally, a responseHeaders object that could be used to override the default cache control among other headers.

Alternatively another function in the same vein as generateMetadata could be created, like generateResponseHeaders, if we want to keep metadata as something focused on head tags.

[victorcarvalhosp]

It's really discouraging that this issue is still open without a fix. This is costing us a lot of money every time Google crawls our website... In my case, it's the exact same issue as: #62294

[ijjk]

Hi, this has now landed in v14.2.10 of Next.js, please upgrade and give it a try!

[cbratschi]

@ijjk still seeing duplicate headers being set: #64864

[ijjk]

@cbratschi duplicate headers sounds like a different issue, this one was about them being overwritten entirely.

[github-actions]

This closed issue has been automatically locked because it had no new activity for 2 weeks. If you are running into a similar issue, please create a new issue with the steps to reproduce. Thank you.