Shouldn't we avoid search engines from indexing our staging environments?

[andrecasal]

Indexing staging environments is a common occurence that is annoying to undo.

As far as I can tell, there's nothing stopping Google from indexing our staging environments.

So shouldn't we stop that with something like this 👇?

// robots[.]txt.ts
import { generateRobotsTxt } from '@nasa-gcn/remix-seo'
import { type LoaderFunctionArgs } from '@remix-run/node'
import { getDomainUrl } from '#app/utils/misc.tsx'

export function loader({ request }: LoaderFunctionArgs) {
	const isProduction = process.env.DEPLOY_STAGE === 'production'
	if(isProduction)
		return generateRobotsTxt([{ type: 'sitemap', value: `${getDomainUrl(request)}/sitemap.xml` }])
	return new Response('User-agent: *\nDisallow: /', { headers: { 'Content-Type': 'text/plain' } })
}

Note we shouldn't use the NODE_ENV var, because we want our stagging environment to be as close to production as possible. The solution is simple: use another var.
DEPLOY_STAGE = "development" | "staging" | "production"

I'd be happy to push a PR for this. LMK :)

[kentcdodds]

We handle this here: #679