atlantis doesnt use default atlantis.yaml unless a project name is specified with preworkflow hooks

[ryan-dyer-sp]

Apologies for the extremely confusing title, so let me explain. Running atlantis v0.16.1

We have our repos.yaml configured to generate the atlantis.yaml via preworkflow hooks:

  repos.yaml: |-
    repos:
    - id: /.*/

      apply_requirements: [approved,mergeable]

      workflow: default

      pre_workflow_hooks:
      - run: terragrunt-atlantis-config generate --autoplan --create-workspace --parallel --output atlantis.yaml --ignore-parent-terragrunt
    
    workflows:
      default:
        plan:
          steps:
          - env:
              name: TERRAGRUNT_TFPATH
              command: 'echo "terraform${ATLANTIS_TERRAFORM_VERSION}"'
          - env:
              name: NODE_PATH
              value: /usr/lib/node_modules
          - run: node ~/terragrunt_light.js plan $PLANFILE
        apply:
          steps:
          - env:
              name: TERRAGRUNT_TFPATH
              command: 'echo "terraform${ATLANTIS_TERRAFORM_VERSION}"'
          - env:
              name: NODE_PATH
              value: /usr/lib/node_modules
          - run: node ~/terragrunt_light.js apply $PLANFILE

Note: the terragrunt_light script is simply an output parser/eliminator.

This generates an atlantis.yaml file in the default directory beneath the PR# directory. atlantis appears to properly create the file and use it when performing the initial atlantis plan. However, if any specific plans are specified (atlantis plan -w <> -d <>) it fails to read all or certain aspects of the atlantis.yaml. Specifically, we are in the process of updating our terraform version which is specified via the terraform_version field at the workspace level in the atlantis.yaml. The individual project atlantis plan is not using the specified version and falling back to the --default-tf-version.

If we update the preworkflow hook to add the project name to the atlantis.yaml, then it appears that atlantis properly reads in the terraform_version as atlantis plan -p <> commands then properly pull in the correct terraform version.

I also reported issue #1487 and am now wondering if the two are somehow related. Now that we have enabled the project name, I will update that ticket as well once we have a few more plan/applies dont with this new config to know if there is any relevance with the project name being required.

@dmattia Just FYI. Seems this may be something you want to at least update docs/how-to/examples on the generator/integration that the project name should probably be set.

[ryan-dyer-sp]

It now turns out we cannot simply do 'atlantis apply' on PRs. We receive the following error:

building command for dir "<path to project>": cannot specify a project name unless an atlantis.yaml file exists to configure projects

[ryan-dyer-sp]

We have worked around this issue by creating a symlink in all of the workspace directories during the plan operation.

    repos:
    - id: /.*/

      apply_requirements: [approved,mergeable]

      workflow: default

      pre_workflow_hooks:
      - run: terragrunt-atlantis-config generate --autoplan --create-workspace --parallel --output atlantis.yaml --ignore-parent-terragrunt &> /dev/null

    workflows:
      default:
        plan:
          steps:
          - env:
              name: TERRAGRUNT_TFPATH
              command: 'echo "terraform${ATLANTIS_TERRAFORM_VERSION}"'
          - env:
              name: NODE_PATH
              value: /usr/lib/node_modules
          - run: bash -c 'ws=$(pwd); while (dirname ${ws} | grep ${WORKSPACE} > /dev/null); do ws=$(dirname ${ws}); done; cd ${ws}; if [ -f ../default/atlantis.yaml ]; then ln -fs ../default/atlantis.yaml; fi'
          - run: node ~/terragrunt_light.js plan $PLANFILE
        apply:
          steps:
          - env:
              name: TERRAGRUNT_TFPATH
              command: 'echo "terraform${ATLANTIS_TERRAFORM_VERSION}"'
          - env:
              name: NODE_PATH
              value: /usr/lib/node_modules
          - run: node ~/terragrunt_light.js apply $PLANFILE

[BoraxTheClean]

It now turns out we cannot simply do 'atlantis apply' on PRs. We receive the following error:

building command for dir "<path to project>": cannot specify a project name unless an atlantis.yaml file exists to configure projects

I have a similar use case and am also experiencing this bug. atlantis apply gives us this error as well.

[msarvar]

pre_workflow_hooks only run in the default workspace because at the time it runs there is no atlantis.yaml. So it makes it hard to run it within a project specific context. Creating a symlink or just copying it over from default folder should work.

[ryan-dyer-sp]

pre_workflow_hooks only run in the default workspace because at the time it runs there is no atlantis.yaml. So it makes it hard to run it within a project specific context. Creating a symlink or just copying it over from default folder should work.

I see what's happening, but it seems that there's a bug in that its happening. if pre_workflow_hooks only run for the folder named after the workflow, then shouldnt the workflow(s) directory be the only directory(s) that atlantis look for when finding the atlantis.yaml? Prior to the hooks, sure the atlantis.yaml would have had to exist in every workflow/workspace as it was part of the repo. But now, this isnt the case and I would argue its a bug.
I dont know the order of operations in how atlantis creates the workspace directories, but if those happen post hook, wouldn't it make more sense for atlantis to just copy the directory instead of doing another checkout from git(if thats even what its doing). And if it does copy the directory post hook, then the atlantis.yaml would exist.
I also totally admit I only know how a terragrunt workflow looks and not how terraform with actual workspaces looks. So if there's something about how other workflows operate that throw a wrench in how I see things, then I'd love to understand them better.

[msarvar]

Pre workflow hooks do not make any assumptions about your atlantis config, or project setup. You can run any arbitrary script with them. They are meant to run once and stop before atlantis get to configure your projects and run plan/policy_check/apply. If pre workflow hook fails Atlantis doesn't stop its workflow execution. You can make you pre workflow hook do the workspace folders setup, once you generate atlantis.yaml copy the cloned repo into workspace specific folders.

Also default workspace is not the same as default workflow. Default workspace is a hardcoded value default that Atlantis clones the initial PR and from there it gets Atlantis config and if needed create workspaces for each project.

On a high level here is execution flow:

PR Created -> Atlantis Receive Webhook -> Execute Pre Workflows -> Build Projects from Atlantis.yaml -> Run autoplan for each project -> (optional) Clone the repo again if projects have a custom workspace and run the plan from there.

We would have to execute pre workflows twice in the execution chain in order to make them re-run for each workspace.

[ryan-dyer-sp]

@msarvar Thanks for the the feedback, its been very helpful. However, I still question, why for the optional clone repo for custom workspaces, is it cloning instead of simply copying what was already cloned? I would think this would make for a faster operation and solve the problem where people are expecting the preworkflow hook to do something which impacts their workspace. We have PRs which create over 100 custom workspaces. That's alot of additional cloning operations. Our use of the preworkflow hook is in creating the atlantis.yaml, so we can't somehow add copying it to all the desired workspaces as they dont exist yet. We figured out our workaround by creating a symlink during the plan.

I still think i'm missing something in the workflow though. During the initial plan for all the individual workspaces, it is using the values from the default workspace atlantis.yaml. During the apply it doesnt. Why is there a difference?

[giuli007]

We have a similar use-case and we have encountered similar issues as well (running v0.17.0).

To clarify the conditions for this to happen are:

• the atlantis.yaml is not part of the repository from which a PR is opened and it is generated on the server-side via pre_workflow_hooks
• some projects in atlantis.yaml are using a non-default workspace

Autoplan (when files in projects are modified) and general atlantis plan work fine.

However there are different combinations of actions that lead to errors

1. Commenting atlantis plan -d <projectdir> -w <workspacename> apparently works but the atlantis.yaml file is ignored and the plan is run with the DefaultProjCfg, which might not be what the end user is expecting
2. Commenting atlantis apply after plans have been generated with autoplan or an explicit atlantis plan or (-p) command fails with

building command for dir "<path to project>": cannot specify a project name unless an atlantis.yaml file exists to configure projects

3. Running atlantis apply -d <projectdir> -w <workspacename> apparently works but the atlantis.yaml file is ignored and the apply is run with the DefaultProjCfg

Here is an example repo-config.yaml to reproduce the issue. For illustration purpose this includes an inline generation of an example atlantis.yaml (we normally use our own script to generate the file dynamically; also terragrunt_wrapper.sh is just our own custom workflow and doesn't matter much for this issue)
https://gist.github.com/giuli007/2a5823722515a6e8b48a4ac77ec48a60

As mentioned in this discussion these errors can be worked around by dynamically linking back to the atlantis.yaml living in the default workspace from non-default ones prior to plan/apply in custom workflows. However I think that is a bit of a hack.

Autoplanning and non-targeted planning (atlantis plan) are doing the right thing (using the atlantis.yaml from the default workspace) and I think it should be possible to fix this.