feat: Enhance Support for CDKTF By Allowing Git Untracked Files to be Added to the Atlantis Modified File List

[X-Guardian]

What

• Add an include-git-untracked-files flag to the Atlantis server configuration.
• Add a GetGitUntrackedFiles function to the server/events/working_dir.go file that runs git ls-files in the relevant working directory and returns a list of untracked files.
• Update the project command builder functions so that if the include-git-untracked-files flag is set, call the GetGitUntrackedFiles function and add any returned files to the list of Atlantis modified files for the plan.
• Update the CDKTF custom workflow documentation to leverage the new include-git-untracked-files flag and remove the requirement to have to commit the CDKTF generated files to the repo.

Other Changes

• Refine userconfig configuration in server/events/project_command_builder_test.go to improve readability.
• Update the DefaultProjectCommandBuilder type property documentation.
• Enhance the debug logging in the server/events/project_command_builder.go file.
• Refine a logging message in server/events/project_finder.go

Why

Closes #3723

Tests

repos.yaml

repos:
- id: github.com/Guardian-Teck/atlantis-cdk
  pre_workflow_hooks:
    - description: CDK TF Init
      run: |
        pnpm install --frozen-lockfile
        pnpm nx affected -t synth --base=HEAD^1 --parallel 1 -- --output ci-cdktf.out

config.yaml

autoplan-file-list: "**/cdk.tf.json"
checkout-strategy: merge
include-git-untracked-files: true

Debug Log Extract

{"level":"debug","ts":"2023-08-31T17:54:27.600+0100","caller":"events/pre_workflow_hooks_command_runner.go:59","msg":"pre-hooks configured, running...","json":{"repo":"Guardian-Teck/atlantis-cdk","pull":"2"}}
{"level":"debug","ts":"2023-08-31T17:54:27.600+0100","caller":"events/pre_workflow_hooks_command_runner.go:65","msg":"got workspace lock","json":{"repo":"Guardian-Teck/atlantis-cdk","pull":"2"}}
{"level":"debug","ts":"2023-08-31T17:54:27.609+0100","caller":"events/pre_workflow_hooks_command_runner.go:109","msg":"Processing pre workflow hook 'CDK TF Init', Command 'plan', Target commands [plan]","json":{"repo":"Guardian-Teck/atlantis-cdk","pull":"2"}}
{"level":"debug","ts":"2023-08-31T17:54:27.609+0100","caller":"events/pre_workflow_hooks_command_runner.go:117","msg":"Running pre workflow hook: 'CDK TF Init'","json":{"repo":"Guardian-Teck/atlantis-cdk","pull":"2"}}
{"level":"debug","ts":"2023-08-31T17:54:27.609+0100","caller":"events/pre_workflow_hooks_command_runner.go:121","msg":"Setting shell to default: \"\"","json":{"repo":"Guardian-Teck/atlantis-cdk","pull":"2"}}
{"level":"debug","ts":"2023-08-31T17:54:27.609+0100","caller":"events/pre_workflow_hooks_command_runner.go:126","msg":"Setting shellArgs to default: \"\"","json":{"repo":"Guardian-Teck/atlantis-cdk","pull":"2"}}
{"level":"info","ts":"2023-08-31T17:54:41.587+0100","caller":"runtime/pre_workflow_hook_runner.go:80","msg":"successfully ran \"sh -c pnpm install --frozen-lockfile\\npnpm nx affected -t synth --base=HEAD^1 --parallel 3 -- --output ci-cdktf.out\\n\" in \"/root/.atlantis/repos/Guardian-Teck/atlantis-cdk/2/default\"","json":{"repo":"Guardian-Teck/atlantis-cdk","pull":"2"}}
{"level":"debug","ts":"2023-08-31T17:54:42.881+0100","caller":"events/project_command_builder.go:269","msg":"Building plan command for all affected projects","json":{"repo":"Guardian-Teck/atlantis-cdk","pull":"2"}}
{"level":"debug","ts":"2023-08-31T17:54:43.118+0100","caller":"events/project_command_builder.go:329","msg":"'include-git-untracked-files' option is set, getting untracked files","json":{"repo":"Guardian-Teck/atlantis-cdk","pull":"2"}}
{"level":"debug","ts":"2023-08-31T17:54:43.118+0100","caller":"events/working_dir.go:395","msg":"Checking for Git untracked files in directory: '/root/.atlantis/repos/Guardian-Teck/atlantis-cdk/2/default'","json":{}}
{"level":"debug","ts":"2023-08-31T17:54:43.129+0100","caller":"events/working_dir.go:405","msg":"Untracked files: 'config/test1/ci-cdktf.out/manifest.json,config/test1/ci-cdktf.out/stacks/test1/.terraform.lock.hcl,config/test1/ci-cdktf.out/stacks/test1/.terraform/providers/registry.terraform.io/hashicorp/local/2.4.0/linux_amd64,config/test1/ci-cdktf.out/stacks/test1/.terraform/terraform.tfstate,config/test1/ci-cdktf.out/stacks/test1/cdk.tf.json,config/test1/ci-cdktf.out/stacks/test1/default.tfplan,config/test2/ci-cdktf.out/manifest.json,config/test2/ci-cdktf.out/stacks/test2/.terraform.lock.hcl,config/test2/ci-cdktf.out/stacks/test2/.terraform/providers/registry.terraform.io/hashicorp/local/2.4.0/linux_amd64,config/test2/ci-cdktf.out/stacks/test2/.terraform/terraform.tfstate,config/test2/ci-cdktf.out/stacks/test2/cdk.tf.json,config/test2/ci-cdktf.out/stacks/test2/default.tfplan,'","json":{}}
{"level":"debug","ts":"2023-08-31T17:54:43.129+0100","caller":"events/project_command_builder.go:337","msg":"33 files were modified in this pull request. Modified files: [.editorconfig .gitignore .npmrc .vscode/extensions.json README.md config/test1/cdktf.json config/test1/main.d.ts config/test1/main.ts config/test1/package.json config/test1/tsconfig.json config/test2/cdktf.json config/test2/main.d.ts config/test2/main.ts config/test2/package.json config/test2/tsconfig.json modules/test/package.json nx.json package.json pnpm-lock.yaml pnpm-workspace.yaml config/test1/ci-cdktf.out/manifest.json config/test1/ci-cdktf.out/stacks/test1/.terraform.lock.hcl config/test1/ci-cdktf.out/stacks/test1/.terraform/providers/registry.terraform.io/hashicorp/local/2.4.0/linux_amd64 config/test1/ci-cdktf.out/stacks/test1/.terraform/terraform.tfstate config/test1/ci-cdktf.out/stacks/test1/cdk.tf.json config/test1/ci-cdktf.out/stacks/test1/default.tfplan config/test2/ci-cdktf.out/manifest.json config/test2/ci-cdktf.out/stacks/test2/.terraform.lock.hcl config/test2/ci-cdktf.out/stacks/test2/.terraform/providers/registry.terraform.io/hashicorp/local/2.4.0/linux_amd64 config/test2/ci-cdktf.out/stacks/test2/.terraform/terraform.tfstate config/test2/ci-cdktf.out/stacks/test2/cdk.tf.json config/test2/ci-cdktf.out/stacks/test2/default.tfplan ]","json":{"repo":"Guardian-Teck/atlantis-cdk","pull":"2"}}
{"level":"debug","ts":"2023-08-31T17:54:43.129+0100","caller":"events/project_command_builder.go:379","msg":"got workspace lock","json":{"repo":"Guardian-Teck/atlantis-cdk","pull":"2"}}
{"level":"debug","ts":"2023-08-31T17:54:43.135+0100","caller":"events/project_command_builder.go:411","msg":"moduleInfo for /root/.atlantis/repos/Guardian-Teck/atlantis-cdk/2/default (matching \"\") = map[]","json":{"repo":"Guardian-Teck/atlantis-cdk","pull":"2"}}
{"level":"info","ts":"2023-08-31T17:54:43.135+0100","caller":"events/project_command_builder.go:463","msg":"found no atlantis.yaml file","json":{"repo":"Guardian-Teck/atlantis-cdk","pull":"2"}}
{"level":"info","ts":"2023-08-31T17:54:43.135+0100","caller":"events/project_finder.go:147","msg":"filtered modified files to 2 file(s) in the autoplan file list: [config/test1/ci-cdktf.out/stacks/test1/cdk.tf.json config/test2/ci-cdktf.out/stacks/test2/cdk.tf.json]","json":{"repo":"Guardian-Teck/atlantis-cdk","pull":"2"}}
{"level":"info","ts":"2023-08-31T17:54:43.135+0100","caller":"events/project_finder.go:172","msg":"there are 2 modified project(s) at path(s): config/test1/ci-cdktf.out/stacks/test1, config/test2/ci-cdktf.out/stacks/test2","json":{"repo":"Guardian-Teck/atlantis-cdk","pull":"2"}}
{"level":"info","ts":"2023-08-31T17:54:43.135+0100","caller":"events/project_command_builder.go:467","msg":"automatically determined that there were 2 projects modified in this pull request: [repofullname=Guardian-Teck/atlantis-cdk path=config/test1/ci-cdktf.out/stacks/test1 repofullname=Guardian-Teck/atlantis-cdk path=config/test2/ci-cdktf.out/stacks/test2]","json":{"repo":"Guardian-Teck/atlantis-cdk","pull":"2"}}
{"level":"debug","ts":"2023-08-31T17:54:43.135+0100","caller":"events/project_command_builder.go:469","msg":"determining config for project at dir: \"config/test1/ci-cdktf.out/stacks/test1\"","json":{"repo":"Guardian-Teck/atlantis-cdk","pull":"2"}}
{"level":"debug","ts":"2023-08-31T17:54:43.135+0100","caller":"events/project_command_context_builder.go:98","msg":"Building project command context for plan","json":{"repo":"Guardian-Teck/atlantis-cdk","pull":"2"}}
{"level":"debug","ts":"2023-08-31T17:54:43.136+0100","caller":"events/project_command_builder.go:469","msg":"determining config for project at dir: \"config/test2/ci-cdktf.out/stacks/test2\"","json":{"repo":"Guardian-Teck/atlantis-cdk","pull":"2"}}
{"level":"debug","ts":"2023-08-31T17:54:43.136+0100","caller":"events/project_command_context_builder.go:98","msg":"Building project command context for plan","json":{"repo":"Guardian-Teck/atlantis-cdk","pull":"2"}}

References

• CDKTF

[jamengual]

so this will only add untracked files on the workspace dir that are generated within that dir, correct? not files outside the workspace dir or dir specified in the workflow?

…

On Thu, Aug 31, 2023 at 10:03 AM Simon Heather ***@***.***> wrote: @X-Guardian <https://github.com/X-Guardian> requested review from @runatlantis/maintainers on: #3724 <#3724> feat: Enhance Support for CDKTF By Allowing Git Untracked Files to be Added to the Atlantis Modified File List as a code owner. — Reply to this email directly, view it on GitHub <#3724 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAQ3ERBM4CHVWR3NX6E3O6LXYC7WFANCNFSM6AAAAAA4GMBCDE> . You are receiving this because your review was requested.Message ID: ***@***.***>

[X-Guardian]

Hi @jamengual, it will run git ls-files --others --exclude-standard in the working directory for the particular repo/pull request/workspace combination for the current plan command. This will report any git untracked files in the working directory that aren't excluded through any .gitignore references.

[GenPage]

LGTM, thank you for the documentation and logging improvements as well