feat: Support loading git token from disk

[meringu]

what

Adds a gh-token-file server setting that can be used instead of gh-token. The token is read from disk as part of the GitHub client transport, allowing the token to be rotated without needing to restart the Atlantis process. I've also re-used the .git-credentials token rotator from the GitHub app integration to ensure that write-git-creds will update the .git-credentials file as the gh-token-file is updated.

why

We run about ~150 Atlantis instances in our organisation in our GitHub org. GitHub have a hard limit on 100 GitHub apps per org, and charge a seat per service account user. To get around these challenges we have developed a GitHub app which issues scoped token for each Atlantis and loads them as Kubernetes secrets. The app is also responsible for forwarding and re-signing the webhooks with per instance webhook secrets to the correct Atlantis instance. I'd potentially be interested in opening the source code for this app if there is interest. There are still a few issues we are working through, like this one, and it is currently in a complicated relationship with Keda.

We run Atlantis as a GitHub app, but configure it with short term credentials to run as a GitHub user. The tokens only last one hour, so we manage restarts as part of scale-to-zero to ensure that Atlantis is always running with a valid token.

If the Atlantis instance fails to restart within an hour due to high activity or long running plan or applies, the commands will finish, but results will fail to be commented back to the GitHub pull request.

With this change we can load the token from disk, and as our GitHub app rotates the token, it is immediately picked up by the running Atlantis instance, allowing it to run uninterrupted for longer periods of time.

tests

I ran an apply with the follow Terraform that used to fail to comment back:

resource "time_sleep" "wait_over_hour" {
  create_duration = "4000s"
}

I also ran some cat commands agains the .git-credentials file with --write-git-creds was specified to ensure that it was getting updated as the token was being rotated.

references