split framework module

[gmalkov]

into terra-cluster and terra-env

[yonghaoy]

LG! Thanks for doing this!
Left few questiosns and minor comment.

[choover-broad]

• For new apps that might be developed over long periods... is the idea that developers maintain a long-lived branch of the terra-env module? Or would we encourage people to add enabled/disabled flags to the new app's module and stay integrated with master? Also, are we anticipating there's a linear version stream on master that is used to push changes from integration -> staging/pre-prod -> prod or will we try to keep them pegged to master and use vars to control rollout of new components instead?

• I wish there was a way to flexibly compose environments with different sets of service modules together! In an ideal world, I could have a configuration file for an environment that specifies the modules I want it to include, the versions for each module, and any vars I want to override. Obviously, that is not how Terraform works 😅

[choover-broad]

One more question - are we in general expecting to run a full copy of Terra in the new dev/integration environments? So it makes sense to include Cromwell resources there, and everything else eventually?

[gmalkov]

These are all really good questions.

Is the idea that developers maintain a long-lived branch of the terra-env module? Or would we encourage people to add enabled/disabled flags to the new app's module and stay integrated with master?

IMO the latter seems cleaner to me. Especially since it looks like we will very soon have count and for_each available for modules, so we won't even need to have flags, we can just have booleans that toggle all the apps in the terra-env module, that can be set from per-environment tfvars files:
hashicorp/terraform#24461

Also, are we anticipating there's a linear version stream on master that is used to push changes from integration -> staging/pre-prod -> prod or will we try to keep them pegged to master and use vars to control rollout of new components instead?

I've been thinking about this. It's tricky because you can't have dynamic evaluation of the module source, so we can't have a variable that dictates a sub-module version. Some possible ways to roll out changes:

• If we can find a clean way to specify a module version per workspace, I would really prefer the approach of a linear version stream of terra-env. One avenue that might be worth investigating is override files. As of now I don't see a clean way to do this.
• If we can't find such a way, we can open a PR and keep it open until the change propagates all the way through the various environments, doing atlantis apply for each environment as we are ready for the changes to come in. After we do prod, merge the PR. This could potentially get messy if there are multiple such long-running PRs open.
• Using tfvars files to roll out new components might currently be the cleanest way to go, especially when that change I linked above gets released.

I wish there was a way to flexibly compose environments with different sets of service modules together!

I think the change I linked will in fact make this possible to implement cleanly, with counts per module we can toggle apps per-environment via tfvars files for terra-env!

[gmalkov]

One more question - are we in general expecting to run a full copy of Terra in the new dev/integration environments? So it makes sense to include Cromwell resources there, and everything else eventually?

As I see it, we will only run a "full copy" of Terra in these environments when all of Terra is ported to k8s and whatever standard of k8s config/deployment tooling we settle on.