Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Disable auto-planning for a specific PR #932

Open
qugu opened this issue Feb 19, 2020 · 4 comments
Open

Disable auto-planning for a specific PR #932

qugu opened this issue Feb 19, 2020 · 4 comments
Labels
feature New functionality/enhancement Stale

Comments

@qugu
Copy link

qugu commented Feb 19, 2020

It would be great to disable auto-planning ad-hoc for a PR when a specific keyword is found in its title or description. Nice to collaborate on something that's never going to be merged.

This is beneficial for draft PRs, but also for those users who do not have this feature due to plan limitations or for those who use other git services.

The 'keyword' could be configurable via any of the YAML files. Atlantis could also behave so based on any other metadata found in the PR's payload (maybe tags?).

@lkysow lkysow added the feature New functionality/enhancement label Feb 19, 2020
minamijoyo added a commit to minamijoyo/atlantis that referenced this issue Sep 8, 2021
This is an attempt to partially support for runatlantis#932.

If an author of the pull request has a confidence of "no changes for
real resources", it would be great if atlantis could skip autoplan.

The initial implementation of this feature as follows:

If a title of pull request contains the following keywords, skip autoplan.

* [skip atlantis]
* [skip ci]
* [atlantis skip]
* [ci skip]

We should always force to invoke plan with explicit comment command.

This feature is currently implemented only for GitHub just because
I'm a user of GitHub, but I expect it's possible to support other VCS
providers.

Note:

Most of general purpose CI/CD platforms support a concept for skip
build. For examples:
https://circleci.com/docs/2.0/skip-build/
https://docs.github.com/en/actions/guides/about-continuous-integration#skipping-workflow-runs

As far as I know, they check all commits included in the pull request,
not a title of the pull request because they need to support triggered
on push event. On the other hand, the current implementation of atlantis
doesn't triggered on push event and doesn't have all commits on open
event. To simplify the implementation, I think checking the title is
reasonable. Of course it's possible to get all commits included in the
pull request dynamically via additional API calls, please let me know if
we should check commit messages instead of the title.

The original feature request said that the 'keyword' could be
configurable, but I don't think most of users including me need such a
flexibility. So the initial implementation embed keywords in source. If
someone want to need it should be configurable, feel free to open
another feature request.
minamijoyo added a commit to minamijoyo/atlantis that referenced this issue Sep 8, 2021
This is an attempt to partially support for runatlantis#932.

If an author of the pull request has a confidence of "no changes for
real resources", it would be great if atlantis could skip autoplan.

The initial implementation of this feature as follows:

If a title of pull request contains the following keywords, skip autoplan.

* [skip atlantis]
* [skip ci]
* [atlantis skip]
* [ci skip]

We should always force to invoke plan with explicit comment command.

This feature is currently implemented only for GitHub just because
I'm a user of GitHub, but I expect it's possible to support other VCS
providers.

Note:

Most of general purpose CI/CD platforms support a concept for skip
build. For examples:
https://circleci.com/docs/2.0/skip-build/
https://docs.github.com/en/actions/guides/about-continuous-integration#skipping-workflow-runs

As far as I know, they check all commits included in the pull request,
not a title of the pull request because they need to support triggered
on push event. On the other hand, the current implementation of atlantis
doesn't triggered on push event and doesn't have all commits on open
event. To simplify the implementation, I think checking the title is
reasonable. Of course it's possible to get all commits included in the
pull request dynamically via additional API calls, please let me know if
we should check commit messages instead of the title.

The original feature request said that the 'keyword' could be
configurable, but I don't think most of users including me need such a
flexibility. So the initial implementation embeds keywords in source. If
someone need to be configurable, feel free to open another feature
request.
@nishkrishnan
Copy link
Contributor

I'm not sure this is a great idea from a security standpoint. On our end, autoplan is used to ensure that an apply must be run (if there are any changes) before any code can be merged. Would like to think about some ideas to mitigate this risk, else I'm inclined to support this.

@minamijoyo
Copy link
Contributor

@nishkrishnan Thank you for your reply! To be honest, my use case is actually different from the original request, I don't think all who needs this feature have the same use case, but let me share my use case.

I'm automating a provider version up workflow with tfupdate, which updates all version constraints in Terraform configurations recursively.
The problem is that I have 200+ root modules and the AWS provider is released weekly :) So I'm running regression tests to ensure that the version up doesn't affect real resources by running terraform plan for all directories in AWS CodeBuild and set a failed status to the PR if it detects any change. So I have full confidence of "no change" except that AWS CodeBuild doesn't have a lock of atlantis. I'm also running daily regression tests on master branch so that unexpected changes will be detected within a day. I'm running regression tests outside of atlantis because it's impractical to make sure that there are no changes in the 200+ directories from the current atlantis outputs. To avoid being overwhelmed by a large number of meaningless comments from atlantis, I'm injecting the following script as a workaround in pre_workflow_hooks which generates an empty atlantis.yml to disable autoplan if the commit message matches some keywords.

if git --no-pager log --no-color --format="%s" | grep -E '(\[skip (atlantis|ci)\])|(\[(atlantis|ci) skip\])' ; then
  echo "generate an empty atlantis.yaml to skip auto plan"
  cat << EOF > atlantis.yaml
version: 3
EOF
fi

If we could plan for wildcard directories (#686) and summarize results (#1267), it would be an ideal solution, but I felt it's too far. After looking for an alternative, I found this (#932). I thought a concept like "[skip ci]" was useful for me and it's probably familiar with others, so I implemented it as #1799.

autoplan is used to ensure that an apply must be run (if there are any changes) before any code can be merged.

I understand it should be true, but there are some loopholes in the current implementations as far as I know. For examples:

(1) Autoplan doesn't detect changes inside module dependences (#920).
(2) We cannot ensure to apply before merge by enforcing a status check for apply (#1316). It highly depends branch protection features of VCS providers, but at least in GitHub, removing mergeable from apply_requirements causes other security risks because approved can be done by only read permission.

On the other hand, The implementation of #1799 only skips autoplan. It doesn't ignore an explicit comment plan command. So If the reviewer can't believe that there are no changes, we can always use the explicit plan command to mitigate the risk,
or is it acceptable if the "skip autoplan" feature is disabled by default?

@minamijoyo
Copy link
Contributor

(2) We cannot ensure to apply before merge by enforcing a status check for apply

Hi @nishkrishnan, I noticed that your patch #1856 in v0.17.5 fixed the problem (2) mentioned above. Thanks!

In addition, @raynigon suggested that the skip keyword should be configurable in #1799 (comment).

If the skip keyword is configurable by server side and doesn't match anything by default, I think it's safe for you, but what do you think of the idea?

minamijoyo added a commit to minamijoyo/atlantis that referenced this issue Sep 22, 2022
This is an attempt to partially support for runatlantis#932.

If an author of the pull request has a confidence of "no changes for
real resources", it would be great if atlantis could skip autoplan.

The initial implementation of this feature as follows:

If a title of pull request contains the following keywords, skip autoplan.

* [skip atlantis]
* [skip ci]
* [atlantis skip]
* [ci skip]

We should always force to invoke plan with explicit comment command.

This feature is currently implemented only for GitHub just because
I'm a user of GitHub, but I expect it's possible to support other VCS
providers.

Note:

Most of general purpose CI/CD platforms support a concept for skip
build. For examples:
https://circleci.com/docs/2.0/skip-build/
https://docs.github.com/en/actions/guides/about-continuous-integration#skipping-workflow-runs

As far as I know, they check all commits included in the pull request,
not a title of the pull request because they need to support triggered
on push event. On the other hand, the current implementation of atlantis
doesn't triggered on push event and doesn't have all commits on open
event. To simplify the implementation, I think checking the title is
reasonable. Of course it's possible to get all commits included in the
pull request dynamically via additional API calls, please let me know if
we should check commit messages instead of the title.

The original feature request said that the 'keyword' could be
configurable, but I don't think most of users including me need such a
flexibility. So the initial implementation embeds keywords in source. If
someone need to be configurable, feel free to open another feature
request.
minamijoyo added a commit to minamijoyo/atlantis that referenced this issue Nov 14, 2022
This is an attempt to partially support for runatlantis#932.

If an author of the pull request has a confidence of "no changes for
real resources", it would be great if atlantis could skip autoplan.

The initial implementation of this feature as follows:

If a title of pull request contains the following keywords, skip autoplan.

* [skip atlantis]
* [skip ci]
* [atlantis skip]
* [ci skip]

We should always force to invoke plan with explicit comment command.

This feature is currently implemented only for GitHub just because
I'm a user of GitHub, but I expect it's possible to support other VCS
providers.

Note:

Most of general purpose CI/CD platforms support a concept for skip
build. For examples:
https://circleci.com/docs/2.0/skip-build/
https://docs.github.com/en/actions/guides/about-continuous-integration#skipping-workflow-runs

As far as I know, they check all commits included in the pull request,
not a title of the pull request because they need to support triggered
on push event. On the other hand, the current implementation of atlantis
doesn't triggered on push event and doesn't have all commits on open
event. To simplify the implementation, I think checking the title is
reasonable. Of course it's possible to get all commits included in the
pull request dynamically via additional API calls, please let me know if
we should check commit messages instead of the title.

The original feature request said that the 'keyword' could be
configurable, but I don't think most of users including me need such a
flexibility. So the initial implementation embeds keywords in source. If
someone need to be configurable, feel free to open another feature
request.
@nitrocode
Copy link
Member

nitrocode commented Dec 22, 2022

Currently if your PR is a draft, new commits will not trigger a plan.

It would be nice to be able to run atlantis plan --no-lock or --skip-lock or something to run a plan without locking.

We support custom switches like --auto-merge-disabled for atlantis apply so I don't see why we couldn't add another custom flag for atlantis plan

autoMergeDisabledFlagLong = "auto-merge-disabled"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New functionality/enhancement Stale
Projects
None yet
Development

No branches or pull requests

5 participants