Allow supplying wildcard for dir

[cyrus-mc]

We define our terraform code in what we call stacks. Each stack defines multiple environments where each directory off root is the environment.

Currently we have to add a project for each directory. Would be nice if you could specify a wildcard for the dir name in the project and just have Atlantis run a plan for each directory that matches that pattern.

[ghost]

I have a project with 60+ subfolder. It will be helpful for me if we have this feature.

[bfanyuk]

Hello! We also looking a way to not duplicate the whole definition for every sub-folder.
Would be very helpful to either allow wildcards or to introduce root-level workspace, apply_requirements and everything else.

[jaswennyhuangg]

Hi, has this feature added in atlantis?

[mjagielka]

Hi guys. Any news here? This option would be really useful if you have lot of subfolders. I found terragrunt-atlantis-config tool to automatically generate atlantis.yaml based on current git repo but it's not enough flexible when you have different workflow's for different sub folders.

[dmattia]

@mjagielka Just a heads up, terragrunt-atlantis-config now supports different workflows for different subfolders by specifying a workflow name in the locals of the parent Terragrunt module 😄

[wmariuss]

Atlantis is applying commands in the path where *.tf file is changed, this happen when i do not have atlantis.yaml file. Why not let the same process in case we have the atlantis.yaml file and i do not have the project specified for a specific path in the config file? Is very hard to maintain atlantis.yaml file in case we have many projects and most of them we apply same version or rules.
So, i can see like this:

• If i do not specify any project in atlantis.yaml file, the process can be default one. Execute or able to execute atlantis plan on any changes from a specific dir without to specify -d
• If i have project in atlantis.yaml file should be the same process as it's now

For me i can see atlantis.yaml file for any custom projects with different config than other.

[djenriquez]

@wmariuss our use case is similar, and this is a common pattern if folks follow the terragrunt model. It is odd that without the repo-level atlantis.yaml, atlantis properly discovers the terraform files that should be planned/applied. However, when you add a repo-level config, all of a sudden it requires that you define every directory subsequent sub-directories. Why is that?

In our use case, we have a need to disable auto-planning but cannot because if we do, we would then have to define every directory and sub-directory since auto-plan disabling is a repo-level config. I understand that terragrunt-atlantis-config can generate this config for us, but our project has different terragrunt version dependencies for different workspaces. There are some that are not compatible with terragrunt-atlantis-config from what I saw (terragrunt v0.18.7).

• Is there a way to specify repo-level config while inheriting the rest of configuration from the server level config?
• @dmattia do you know if terragrunt-atlantis-config can be run to support a project that implements different versions of terragrunt?

[dmattia]

@djenriquez feel free to move this discussion over to https://github.com/transcend-io/terragrunt-atlantis-config if you'd like to keep this Issue on topic 😄. But the short answer is that I think it is possible to use different versions of terragrunt:

For each version of terragrunt you use, you could define a custom atlantis workflow, and then on each terragrunt module add a atlantis_workflow local value that names the workflow it should use. To make this scalable, you can also specify the atlantis_workflow in a parent terragrunt file, and all child terragrunt modules will then inherit and use that workflow.

[yasra002]

@mjagielka Just a heads up, terragrunt-atlantis-config now supports different workflows for different subfolders by specifying a workflow name in the locals of the parent Terragrunt module 😄

Hi @dmattia!

Will it be possible to share an example of how to specify workflow name in the locals. Is it something like below.

locals {
workflow = "someworkflow"
}

[mjagielka]

Hey @yasra002

locals {
  atlantis_workflow = "someworkflow"
}

• add your workflow to atlantis.yaml

workflows:
  someworkflow:
    apply:
      steps:
      .......

[yasra002]

Hey @yasra002

locals {
  atlantis_workflow = "someworkflow"
}

• add your workflow to atlantis.yaml

workflows:
  someworkflow:
    apply:
      steps:
      .......

Thanks. I managed to make it work. I did not want to add workflows to atlantis.yaml so instead, I added those to Server side repo config.

[jamengual]

is this still an issue with v0.19.8?

[norman-zon]

@jamengual: Yes the issue still persists.
Doing something like

# atlantis.yaml
version: 3
projects:
- dir: dir1/**
  workflow: default
- dir: dir2/**
  workflow: terragrunt

still does not work

[jamengual]

I do not think this is even a feature

[edalford11]

I do not think this is even a feature

This issue is requesting the feature be added. Would really simplify the configuration if we can use this approach.

[sahaqaa]

Hi, will this feature going to be added in Atlantis?

[te-ak]

Please add this feature

[nitrocode]

There are a couple prs that have tried to tackle this and did not know there is a wildcard regex for projects via --enable-regexp-cmd and then being able to run atlantis plan -p .*. We want to do the same with -d.

Here are the prs

• feat: implement plan-all command #2694
• feat: Add pattern matching support to atlantis plan/apply -d flag #2742
• Previous pr for the -p command Support regexp in command builder on the project name #1419
• This ticket seems like a duplicate of Allow regexp command for -d and -w flags: #259.

[Fabianoshz]

@nitrocode I believe we have 2 different things:

• Adding regex support in the projects.dir on the repo config
• Adding regex suppor for the plan command.

The first should add more control from the admin side while the latter should add more control from the client side.

Looking at this code:

atlantis/server/core/config/raw/project.go

Lines 81 to 126 in c800f70

	func (p Project) ToValid() valid.Project {
	var v valid.Project
	// Prepend ./ and then run .Clean() so we're guaranteed to have a relative
	// directory. This is necessary because we use this dir without sanitation
	// in DefaultProjectFinder.
	cleanedDir := filepath.Clean("./" + *p.Dir)
	v.Dir = cleanedDir
	if p.Branch != nil {
	branch := *p.Branch
	withoutSlashes := branch[1 : len(branch)-1]
	// Safe to use MustCompile because we test it in Validate().
	v.BranchRegex = regexp.MustCompile(withoutSlashes)
	}
	if p.Workspace == nil || *p.Workspace == "" {
	v.Workspace = DefaultWorkspace
	} else {
	v.Workspace = *p.Workspace
	}
	v.WorkflowName = p.Workflow
	if p.TerraformVersion != nil {
	v.TerraformVersion, _ = version.NewVersion(*p.TerraformVersion)
	}
	if p.Autoplan == nil {
	v.Autoplan = DefaultAutoPlan()
	} else {
	v.Autoplan = p.Autoplan.ToValid()
	}
	// There are no default apply requirements.
	v.ApplyRequirements = p.ApplyRequirements
	v.Name = p.Name
	if p.DeleteSourceBranchOnMerge != nil {
	v.DeleteSourceBranchOnMerge = p.DeleteSourceBranchOnMerge
	}
	if p.ExecutionOrderGroup != nil {
	v.ExecutionOrderGroup = *p.ExecutionOrderGroup
	}
	return v
	}

Seems to me that if we change this function to return multiple projects based on the regex received in the v.Dir we might be able to accept configs like this:

# atlantis.yaml
version: 3
projects:
- dir: dir1/**
  workflow: default
- dir: dir2/**
  workflow: terragrunt

This would work like an expansion of projects internally based on the received regex. The downside is that the internally mapped RepoCfg struct would be different from the repo config file.

Another option to accept a regex in the repo config would be to change all functions that uses the project.dir use regex to match all the possible files. The downside is that I believe this would require a lot of work.

I might give a try to add regex support on the projects.dir repo config, what you think? And which way do you think would be better, expanding the projects or change all the functions that uses the project.dir?

[nitrocode]

Adding regex support in the projects.dir on the repo config

I think this would be difficult to manage code wise and configuration wise.

Adding regex suppor for the plan command.

I'd prefer this for now. We can be explicit about our directories in atlantis.yaml and then use regex to do plans from the PR. I think this gives us the best of both worlds.

cc: @runatlantis/maintainers for more thoughts on this

[vl-kp]

we would like to have this feature otherwise there will be duplicated configs for multi dirs

[rednap]

any progress on this issue? would love to use atlantis with our existing infra, but would need 500+ lines in a repo level config.

[nitrocode]

If there aren't any linked PRs then most likely there isn't any progress on the issue.

@rednap please feel free to contribute the change. All contributors are welcome.

[nitrocode]

One easy way of implementing this would be to default an autodiscovered (so not explicitly defined) project's name to its root dir

cc: @jskrill @runatlantis/core-contributors @runatlantis/maintainers

[gespi1]

wildcards/regex on dir makes sense

[gustysap]

hope this feature will be there soon