RFC: introducing environments

[xtuc]

Problem

As described #199, we should have a way to configure Wrangler based on an environment name.

Proposal

New command line argument

I'm suggesting to add an --env NAME argument for wrangler publish and wrangler build. Additionally we could use an environment variable.

The build commande is included because a custom webpack configuration could take advantage of it to provide a specific configuration. However, to be able to access it we need to inject an environment variable called WRANGLER_ENV.

Segmented configuration

I think that the entire configuration needs to be segmented by env, like the following example:

[env.prod]
name = "wasm-worker-prod"
zone_id = "zone-prod"
account_id = "account-prod"
route = "https://prod.site"
kv-namespaces = ...

[env.test]
name = "wasm-worker-test"
zone_id = "zone-test"
account_id = "account-test"
route = "https://test.site"
kv-namespaces = ...

Some people expressed concern about kv-namespaces being common to all environment, using that kind of configuration it can be different in each environment while maintaining the same JavaScript interface.

Why arbitrary env?

I doubt we know everyone's workflow, we should let the user make Wrangler fit to their needs.

While we can't indentify which env is for prod (optimized) and dev (debug) I don't think it's an issue; Cloudflare's Worker is just a compilation target I think we should build for prod all the time. For debugging we could take advantage of source maps (or Wasm equivalent, if any).

[gabbifish]

Instead of kv-namespaces, could we even more broadly allow people to specify production vs staging/test bindings? This could be useful for testing changes to bindings (e.g. WASM) in staging before moving to production.

[exvuma]

This is a must! What about incorporating this in wrangler config for tags instead of in the toml?
Workflow to set up would be

wrangler config global <api-key> <email>
wrangler config zone -name v.workers.dev -zone_tag <tag> -account_tag <tag> #zone_tag optional
#or could also just edit their  ~/.wrangler/config/zones.toml

workflow to use the config

wrangler init env -name dev -zone v.workers.dev
#then goes in to toml to add routes, kv for that env..
wrangler publish -env dev

I know this is more work, but this would prevent users from having to copy/paste their wrangler.toml everywhere and then getting account and zone tags becomes part of the once per user task versus a once per project

[ashleymichal]

re: @gabbifish 's comment, i think adding configuration for wasm modules would be a great add independent of this change; i prefer a world in which users don't think about bindings (an implementation detail of the workers config api), but about wasm modules and kv namespaces.

re: @victoriabernard92 's comment, long cli commands are rather cumbersome, i for sure prefer adding lines to config files, even if it means copy-pasting, or cping files and editing them. The approach in this RFC could be extended later with support for "composing" your toml the way that serverless supports composing their yaml; it isn't strictly necessary, but is a nice to have for those who prefer their configs to stay pretty DRY.

the premise of #302 appeals to me (specify .toml) as an alternative, but only based on taste; i think both designs could be enhanced by the composition approach above, it just comes down to whether you prefer to have a single toml containing all of your configs, or a separate toml per "environment".

re: kv_namespaces, do recall the updated design for how this should look in the toml ( #319 ). I can for sure see a world in which you have "staging" and "prod" kv namespaces that use the same binding but bind to different namespace ids based on env.

[EverlastingBugstopper]

I think #302 as an alternative to this would be a good start, but is less appealing to me as if you had a couple of different environments, you would end up cluttering your development directory (I can also see naming conventions for different wrangler.toml files getting really out of hand)

One outstanding question I have for this RFC is how do we want to handle deploying to zoneless workers with this environment setup? I can imagine a world where people stage on staging.sub.workers.dev and then want to also publish the same script to production on example.com -- how do we configure zoneless workers in this config? Perhaps we just eliminate the zone_id from the config + infer where to deploy from the route? (I'm also not super familiar with the zoneless API, please chime in + correct me)

re @victoriabernard92 I don't think we want to have a giant command like that for configuring your zones - I think it almost makes more sense to pull down the available zones for an account from the API or to have another type of configuration file that allows you to alias zone/account combos

[zone.prod]
zone_id = "zone-prod"
account_id = "account-prod"

[zone.staging]
zone_id = "zone-staging"
account_id = "account-staging"

^ This also has the same issue of - how do we do stuff w/zoneless

[ashleymichal]

@EverlastingBugstopper in the short term, i assume the same rules apply as for when you are publishing with wrangler today: if you pass the --release flag we will try to deploy on your zone with routes, else we will only deploy to workers.dev.

changing that convention may not be within the scope of this rfc?

[exvuma]

My suggestion had two parts I should've broken down:

1. Account and Zone IDs should live somewhere we can reference for all projects
2. Long command lines to generate .toml
   I agree that 2 is bad design I was more just trying to illustrate 1.

I bring up 1. in this RFC because the global config toml's should be designed to mirror how the wrangler.toml is configured for environments. I love @EverlastingBugstopper's idea of pulling the Zones down to form
~/.wrangler/config/zones.toml

[zone.my.com]
zone_id = "zone-prod"
account_id = "account-prod"

[zone.v.workers.dev]
account_id = "account-prod"

Thus significantly reduces "noise" ~/project/wrangler.toml while not sacrificing any "local context" (stole from Explicit Article 🤓) important to the user

[script.name]
'private':true
'routes': 'my.com/blah*' 'other.com/foo'

[script.dev]
'private':false

Note name is really just the name of the script. Because routes require full zone name's the zone and account ID can be inferred

[EverlastingBugstopper]

So it looks like there are a couple of things going on with this RFC. At the most basic level there is a desire to deploy a single project to multiple zones (and maybe across separate accounts, though that is less of a priority).

There seem to be a couple of approaches that we can take:

1. Have a single wrangler.toml file per project with multiple environments where you specify everything needed for the deploy (account id, zone id, routes, bindings). This has the advantage of being relatively easy to implement, and for developers to easily see where exactly this project is being deployed. This design would require zone and account IDs to be copied between project wrangler.toml files, which @victoriabernard92 has pointed out is quite tedious, and would likely lead to a lot of duplicated IDs, even within the same project.
2. Have a single wrangler.toml file per project with multiple environments in addition to a zones.toml in ~/.wrangler/config. The zones.toml would allow you to link zones to accounts and name them. The wrangler.toml would then be made simpler as developers could simply reference the name of the zone in their zones.toml instead of listing their account id and zone id. This is more difficult to implement and will likely require further design as to the layout of zones.toml and the commands used to generate and manage it.
3. Both of the above proposals could be modified to make them composable -- separate different environments into different .toml files. I personally don't think that this is necessary and introduces a lot of complexity that is unnecessary. Willing to discuss though if anybody feels super strongly about it.

Personally I think that solution 1 is perfectly fine and while you will need to copy + paste your configurations for new projects, it's generally easier than solution 2. I find solution 2 to be cumbersome to implement and also would be hard for developers who are working on a shared account as they wouldn't be able to clone a repo with the account/zone ids filled in already in the wrangler.toml and would need to configure separately.

[exvuma]

hard for developers who are working on a shared account as they wouldn't be able to clone a repo with the account/zone ids

Since they have to run wrangler config anyways, the account tag, zone tags and KV will all be populated in that global file. Committing the script.name will still share the tags

This isn't fully flushed out for how to touch workers.dev, kv,.. that the configured account doesn't own. I imagine modeling it after the endpoint /client/v4/memberships. I agree may be too much implementation costs. Not having to go to the UI to get all tags (especially for KV it's uber annoying) would be a huge win for me personally because I manage a very large amount of accounts/zones/KVs but maybe I am not average "/me shurgs".

[ashleymichal]

@victoriabernard92 listing your kv namespaces is part of milestone 6 (KV commands) so you will no longer need to go to the ui, if that makes a difference. #342

[EverlastingBugstopper]

I think maybe it makes sense to start with solution 1 and then maybe later we can move to solution 2 if it makes sense/we don't think the proposed solution is ergonomic enough? Open to discussing this more.

[ashleygwilliams]

let's start implementing solution 1! once we get that in place we can think about how we'd extend it.

[EverlastingBugstopper]

Closing in favor of #385