feat: allow setting runners.docker.services (#491)

* fix: Pass token to metadata service requests. close: #476 * docs: fix generation tf docs * docs: auto update terraform docs * feat: allow setting runners.docker.services. #489 * add description to new variable * Comment out docker service in example. Improve docs * remove accidentally added file Co-authored-by: Niek Palm <[email protected]> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
cattle-ops · Dec 11, 2022 · 6d73e99 · 6d73e99
1 parent bcb0c0e
commit 6d73e99
Show file tree

Hide file tree

Showing 7 changed files with 86 additions and 29 deletions.
diff --git a/examples/runner-default/README.md b/examples/runner-default/README.md
@@ -8,7 +8,8 @@ This examples shows:
   - You can log into the instance via SSM (Session Manager).
   - Registration via GitLab token.
   - Auto scaling using `docker+machine` executor.
-  - Addtional security groups that are allowed access to the runner agent
+  - Additional security groups that are allowed access to the runner agent
+  - Use of `runners.docker.services` to configure docker registry mirror (commented out - uncomment to apply)
 
 ![runners-default](https://github.com/npalm/assets/raw/main/images/terraform-aws-gitlab-runner/runner-default.png)
 
@@ -41,47 +42,47 @@ No output.
 <!-- BEGIN_TF_DOCS -->
 ## Requirements
 
-| Name | Version |
-|------|---------|
-| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 1 |
-| <a name="requirement_aws"></a> [aws](#requirement\_aws) | ~> 4.7 |
-| <a name="requirement_local"></a> [local](#requirement\_local) | ~> 2 |
-| <a name="requirement_null"></a> [null](#requirement\_null) | ~> 3.0 |
-| <a name="requirement_random"></a> [random](#requirement\_random) | ~> 3.0 |
-| <a name="requirement_tls"></a> [tls](#requirement\_tls) | ~> 3 |
+| Name                                                                      | Version |
+| ------------------------------------------------------------------------- | ------- |
+| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 1    |
+| <a name="requirement_aws"></a> [aws](#requirement\_aws)                   | ~> 4.7  |
+| <a name="requirement_local"></a> [local](#requirement\_local)             | ~> 2    |
+| <a name="requirement_null"></a> [null](#requirement\_null)                | ~> 3.0  |
+| <a name="requirement_random"></a> [random](#requirement\_random)          | ~> 3.0  |
+| <a name="requirement_tls"></a> [tls](#requirement\_tls)                   | ~> 3    |
 
 ## Providers
 
-| Name | Version |
-|------|---------|
-| <a name="provider_aws"></a> [aws](#provider\_aws) | ~> 4.7 |
-| <a name="provider_null"></a> [null](#provider\_null) | ~> 3.0 |
+| Name                                                 | Version |
+| ---------------------------------------------------- | ------- |
+| <a name="provider_aws"></a> [aws](#provider\_aws)    | ~> 4.7  |
+| <a name="provider_null"></a> [null](#provider\_null) | ~> 3.0  |
 
 ## Modules
 
-| Name | Source | Version |
-|------|--------|---------|
-| <a name="module_runner"></a> [runner](#module\_runner) | ../../ | n/a |
-| <a name="module_vpc"></a> [vpc](#module\_vpc) | terraform-aws-modules/vpc/aws | 2.70 |
+| Name                                                   | Source                        | Version |
+| ------------------------------------------------------ | ----------------------------- | ------- |
+| <a name="module_runner"></a> [runner](#module\_runner) | ../../                        | n/a     |
+| <a name="module_vpc"></a> [vpc](#module\_vpc)          | terraform-aws-modules/vpc/aws | 2.70    |
 
 ## Resources
 
-| Name | Type |
-|------|------|
-| [null_resource.cancel_spot_requests](https://registry.terraform.io/providers/hashicorp/null/latest/docs/resources/resource) | resource |
+| Name                                                                                                                                  | Type        |
+| ------------------------------------------------------------------------------------------------------------------------------------- | ----------- |
+| [null_resource.cancel_spot_requests](https://registry.terraform.io/providers/hashicorp/null/latest/docs/resources/resource)           | resource    |
 | [aws_availability_zones.available](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/availability_zones) | data source |
-| [aws_security_group.default](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/security_group) | data source |
+| [aws_security_group.default](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/security_group)           | data source |
 
 ## Inputs
 
-| Name | Description | Type | Default | Required |
-|------|-------------|------|---------|:--------:|
-| <a name="input_aws_region"></a> [aws\_region](#input\_aws\_region) | AWS region. | `string` | `"eu-west-1"` | no |
-| <a name="input_environment"></a> [environment](#input\_environment) | A name that identifies the environment, will used as prefix and for tagging. | `string` | `"runners-default"` | no |
-| <a name="input_gitlab_url"></a> [gitlab\_url](#input\_gitlab\_url) | URL of the gitlab instance to connect to. | `string` | `"https://gitlab.com"` | no |
-| <a name="input_registration_token"></a> [registration\_token](#input\_registration\_token) | n/a | `any` | n/a | yes |
-| <a name="input_runner_name"></a> [runner\_name](#input\_runner\_name) | Name of the runner, will be used in the runner config.toml | `string` | `"default-auto"` | no |
-| <a name="input_timezone"></a> [timezone](#input\_timezone) | Name of the timezone that the runner will be used in. | `string` | `"Europe/Amsterdam"` | no |
+| Name                                                                                       | Description                                                                  | Type     | Default                | Required |
+| ------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------- | -------- | ---------------------- | :------: |
+| <a name="input_aws_region"></a> [aws\_region](#input\_aws\_region)                         | AWS region.                                                                  | `string` | `"eu-west-1"`          |    no    |
+| <a name="input_environment"></a> [environment](#input\_environment)                        | A name that identifies the environment, will used as prefix and for tagging. | `string` | `"runners-default"`    |    no    |
+| <a name="input_gitlab_url"></a> [gitlab\_url](#input\_gitlab\_url)                         | URL of the gitlab instance to connect to.                                    | `string` | `"https://gitlab.com"` |    no    |
+| <a name="input_registration_token"></a> [registration\_token](#input\_registration\_token) | n/a                                                                          | `any`    | n/a                    |   yes    |
+| <a name="input_runner_name"></a> [runner\_name](#input\_runner\_name)                      | Name of the runner, will be used in the runner config.toml                   | `string` | `"default-auto"`       |    no    |
+| <a name="input_timezone"></a> [timezone](#input\_timezone)                                 | Name of the timezone that the runner will be used in.                        | `string` | `"Europe/Amsterdam"`   |    no    |
 
 ## Outputs
 

diff --git a/examples/runner-default/main.tf b/examples/runner-default/main.tf
@@ -94,6 +94,37 @@ module "runner" {
   EOT
 
   runners_post_build_script = "\"echo 'single line'\""
+
+  # Uncomment the HCL code below to configure a docker service so that registry mirror is used in auto-devops jobs
+  # See https://gitlab.com/gitlab-org/gitlab-runner/-/issues/27171 and https://docs.gitlab.com/ee/ci/docker/using_docker_build.html#the-service-in-the-gitlab-runner-configuration-file
+  # You can check this works with a CI job like:
+  # <pre>
+  # default:
+  #    tags:
+  #        - "docker_spot_runner"
+  # docker-mirror-check:
+  #    image: docker:20.10.16
+  #    stage: build
+  #    variables: 
+  #        DOCKER_TLS_CERTDIR: ''
+  #    script:
+  #        - |
+  #        - docker info
+  #          if ! docker info | grep -i mirror
+  #            then
+  #              exit 1
+  #              echo "No mirror config found"
+  #          fi
+  # </pre>
+  #
+  # If not using an official docker image for your job, you may need to specify `DOCKER_HOST: tcp://docker:2375`
+  ## UNCOMMENT 6 LINES BELOW
+  # runners_docker_services = [{
+  #   name       = "docker:20.10.16-dind"
+  #   alias      = "docker"
+  #   command    = ["--registry-mirror", "https://mirror.gcr.io"]
+  #   entrypoint = ["dockerd-entrypoint.sh"]
+  # }]
 }
 
 resource "null_resource" "cancel_spot_requests" {

diff --git a/locals.tf b/locals.tf
@@ -34,6 +34,11 @@ locals {
     }
   )
 
+  runners_docker_services = templatefile("${path.module}/template/runners_docker_services.tpl", {
+    runners_docker_services = var.runners_docker_services
+    }
+  )
+
   runners_pull_policies = var.runners_pull_policy != "" ? "[\"${var.runners_pull_policy}\"]" : "[\"${join("\",\"", var.runners_pull_policies)}\"]"
 
   /* determines if the docker machine executable adds the Name tag automatically (versions >= 0.16.2) */

diff --git a/main.tf b/main.tf
@@ -126,6 +126,7 @@ locals {
       runners_check_interval            = var.runners_check_interval
       runners_volumes_tmpfs             = join("\n", [for v in var.runners_volumes_tmpfs : format("\"%s\" = \"%s\"", v.volume, v.options)])
       runners_services_volumes_tmpfs    = join("\n", [for v in var.runners_services_volumes_tmpfs : format("\"%s\" = \"%s\"", v.volume, v.options)])
+      runners_docker_services           = local.runners_docker_services
       bucket_name                       = local.bucket_name
       shared_cache                      = var.cache_shared
       sentry_dsn                        = var.sentry_dsn

diff --git a/template/runner-config.tpl b/template/runner-config.tpl
@@ -28,6 +28,7 @@ listen_address = "${prometheus_listen_address}"
     pull_policy = ${runners_pull_policies}
     runtime = "${runners_docker_runtime}"
     helper_image = "${runners_helper_image}"
+    ${runners_docker_services}
   [runners.docker.tmpfs]
     ${runners_volumes_tmpfs}
   [runners.docker.services_tmpfs]

diff --git a/template/runners_docker_services.tpl b/template/runners_docker_services.tpl
@@ -0,0 +1,7 @@
+%{ for config in runners_docker_services ~}
+[[runners.docker.services]]
+      name = "${config.name}"
+      alias = "${config.alias}"
+      entrypoint = [${replace(format("\"%s\"", join("\",\"", config.entrypoint)), "/\"{2,}/", "\"")}]
+      command = [${replace(format("\"%s\"", join("\",\"", config.command)), "/\"{2,}/", "\"")}]
+%{ endfor ~}
diff --git a/variables.tf b/variables.tf
@@ -696,6 +696,17 @@ variable "runners_services_volumes_tmpfs" {
   default = []
 }
 
+variable "runners_docker_services" {
+  description = "adds `runners.docker.services` blocks to config.toml.  All fields must be set (examine the Dockerfile of the service image for the entrypoint - see ./examples/runner-default/main.tf)"
+  type = list(object({
+    name       = string
+    alias      = string
+    entrypoint = list(string)
+    command    = list(string)
+  }))
+  default = []
+}
+
 variable "kms_key_id" {
   description = "KMS key id to encrypted the CloudWatch logs. Ensure CloudWatch has access to the provided KMS key."
   type        = string