[Feature request] Add support for "pipeline" > "Environments"

[DamonStamper]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

Add support for a azuredevops_environment resource.

Ideally support defining an environment, ignore deploys made to it, and defining "approvals and checks".

New or Affected Resource(s)

• azuredevops_environment

[xuzhang3]

Hi @DamonStamper , Environments API is supported at v6.0, Current sdk API version is 5.1. This resource will pending until auzde-deviops-sdk support it. SDK issue

[Marcus-James-Adams]

Is there a time scale for v6.0 of the SDK? Our pipelines rely heavily on this.

[xuzhang3]

Azure-DevOps-GO-SDK is managed by another team, they have working on it, but don't have a release date.

[DevOpsBoondoggles]

This would definitely be great since it's the only way to do manual approval stages via yaml. Appreciate the dependency is not from you though.

[picardsrcd]

This would definitely be great since it's the only way to do manual approval stages via yaml. Appreciate the dependency is not from you though.

An interim approach is possible with null-resource
https://joefecht.com/posts/creating-an-azdo-environment-via-api/

// Create Environment using API
// https://github.com/microsoft/terraform-provider-azuredevops/issues/143
resource "null_resource" "environment" {
  provisioner "local-exec" {
    command = "./create_environment.sh"

    environment = {
      azdo_pat_token_owner_email = var.azdo_pat_token_owner_email
      azdo_pat_token             = var.azdo_pat_token
      project_url                = module.azdo.azdo_project_url

      name = "core_infrastructure_env"
    }
  }
}

# This script to call the API is required because of
# https://github.com/microsoft/terraform-provider-azuredevops/issues/143
# The Go SDK is not updated yet to interact with the API V6.0

# This is based on content from https://joefecht.com/posts/creating-an-azdo-environment-via-api/
# Payload Example:
# https://github.com/jf781/AzureDevopsEnvironments/blob/master/payload-samples/env.json

curl -X POST \
    -u "${azdo_pat_token_owner_email}:${azdo_pat_token}" \
    -H "Content-Type: application/json" \
    --data "{\"name\":\"${name}\",\"description\":\"${name} environment\"}" "${project_url}/_apis/distributedtask/environments?api-version=5.0-preview.1"
    
# Note: If the environment exists, the following message will show.
# null_resource.environment (local-exec): {"$id":"1","innerException":null,"message":"Environment 'core_infrastructure_env' already exists in current project.","typeName":"Microsoft.TeamFoundation.DistributedTask.WebApi.EnvironmentExistsException, Microsoft.TeamFoundation.DistributedTask.WebApi","typeKey":"EnvironmentExistsException","errorCode":0,"eventId":3000}

[richardgavel]

I know it sounds ugly, but do we have to use the Go SDK, what about direct REST calls from within the TF provider? I only ask because it's been 5 months since the GO API has been updated in Github, even though it looks like these are all generated files.

[whiskeysierra]

Honestly, both the go SDK and the terraform provider are maintained by Microsoft. I don't see a reason why this hasn't been done yet...

…

On Tue, 26 Jan 2021, 16:10 Richard Gavel, ***@***.***> wrote: I know it sounds ugly, but do we have to use the Go SDK, what about direct REST calls from within the TF provider? I only ask because it's been 5 months since the GO API has been updated in Github, even though it looks like these are all generated files. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#143 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADI7HMUGI3LT7PQDDJHNNDS33LOXANCNFSM4PZ6635Q> .

[xuzhang3]

Hi @richardgavel @whiskeysierra Terraform and SDK are managed by different team, we have feedback this to the SDK team.

[whiskeysierra]

@xuzhang3 I'm in no position to suggest how Microsoft conducts their (open source) business, but as a user I see they represent themselves as a company (notably an organization on github: https://github.com/microsoft) and not as a bunch of teams. Obviously you need to have an organizational structure to run a company, but I fail to see why that is relevant to the outside. Feels like a classic case of "Don't ship your org chart" to me, honestly.

[richardgavel]

I will say this. This is an open source project that accepts contributions. I'm sure the devs that own this repo have a lot to do. But given that, ideally they would first focus on removing blockers to contributions (i.e. reviewing PRs promptly, the old SDK issue) so we can add capabilities on our own rather than demanding something of a free library (that is in support of something MS doesn't even own: Terraform). My hope is that the ADO SDK refresh might be more likely with one of their own pushing the team as opposed to a Github issue.

[avishnyakov]

Might be a reasonable approach to go forward using REST API for the time being, good suggestion by @richardgavel

Go SDK repo seems to be supported by a single person (fair share of commits by me person, majority). Not much been happening there for quite a time. Best move forward and then refactor once SDK catches up?

[whiskeysierra]

For those who want to get something up and running right now: The REST API provider is an acceptable workaround (for me at least) right now. It's reasonably short and configurable enough to allow even updates/replacements, i.e. it properly tracks the lifecycle of the environment resource.

Example

resource "restapi_object" "environment" {
  path = "/${var.project.name}/_apis/distributedtask/environments"
  data = jsonencode({
    name        = var.name
    description = "Managed by Terraform"
  })
  id_attribute = "id"
  update_method = "PATCH"
}

The necessary settings for the provider block:

provider "restapi" {
  uri                  = "https://dev.azure.com/${var.org}"
  username             = "could-be-anything-but-must-not-be-empty"
  write_returns_object = true
  headers = {
    "Accept" = "application/json;api-version=6.1-preview.1"
  }
  # TODO API_DATA_IS_SENSITIVE env var?
}

You could pass the password (PAT in our case) to the provider block directly. I used the env var instead (example here is using a DevOps YAML pipeline):

- task: CmdLine@2
  # ...
  env:
    AZDO_PERSONAL_ACCESS_TOKEN: $(System.AccessToken)
    REST_API_PASSWORD: $(System.AccessToken)

I used the same technique for the Kubernetes Environment/Resource (https://docs.microsoft.com/en-us/rest/api/azure/devops/distributedtask/kubernetes?view=azure-devops-rest-6.1). Works like a charm so far:

# Hard to find a good name for this:
#  - DevOps UI calls it "Resource"
#  - DevOps API calls it "Kubernetes" provider
#  - Service Connection Link (https://joefecht.com/posts/creating-an-azdo-environment-via-api/)
resource "restapi_object" "resource" {
  path = "/${var.project.name}/_apis/distributedtask/environments/${var.environment.id}/providers/kubernetes"
  data = jsonencode({
    name              = local.application.name
    namespace         = kubernetes_namespace.default.metadata[0].name
    clusterName       = var.k8s.name
    serviceEndpointId = azuredevops_serviceendpoint_kubernetes.default.id
  })
  id_attribute = "id"
  force_new = [
    "/name",
    "/namespace",
    "/clusterName",
    "/serviceEndpointId"
  ]
}

(The service endpoint/connection for kubernetes is managed with the DevOps provider directly.)

References

• https://joefecht.com/posts/creating-an-azdo-environment-via-api/
• https://docs.microsoft.com/en-us/rest/api/azure/devops/distributedtask/environments/update?view=azure-devops-rest-6.1
• https://registry.terraform.io/providers/KirillMeleshko/restapi/latest

[whiskeysierra]

Slight fix: I used force_new incorrectly there. It's not a list of field names/paths, but rather a list of strings that force a replacement if they change. I ended up putting all my variables in there which make up the body so if any of them changes a new resource is being created (since PUT/PATCH are not supported).

[whiskeysierra]

I also stumbled across this issue, which people might also face that want to use service account authorization for kubernetes service endpoints. I'm just leaving the link here for reference: https://developercommunity2.visualstudio.com/t/unable-to-create-a-workable-kubernetes-service-con/1337297

[whiskeysierra]

Related to the comment above, here is the restapi_object resource to create a kubernetes service connection with service account authorization:

# Needed because azuredevops_serviceendpoint_kubernetes doesn't allow to change acceptUntrustedCerts
resource "restapi_object" "service-connection" {
  path = "/${var.project.name}/_apis/serviceendpoint/endpoints"
  data = jsonencode({
    name        = "${var.k8s.name}/${local.application.name}"
    description = "Managed by Terraform"
    type        = "kubernetes"
    url         = var.k8s.url
    authorization = {
      scheme = "Token"
      parameters = {
        apitoken                  = base64encode(local.service_account_token)
        serviceAccountCertificate = base64encode(local.service_account_certificate)
      }
    }
    data = {
      authorizationType    = "ServiceAccount"
      acceptUntrustedCerts = "true"
    }
    serviceEndpointProjectReferences = [
      {
        name        = "${var.k8s.name}/${local.application.name}"
        description = "Managed by Terraform"
        projectReference = {
          id   = var.realm-stage.project.id
          name = var.realm-stage.project.name
        }
      }
    ]
  })
  id_attribute = "id"
}

[rahmnstein]

Nice, got it working with restapi_object, do anyone know if there is an API for adding approvals to an environment? I cannot find it, maybe it does not exist yet.

[whiskeysierra]

You can always perform the action in the UI and inspect network requests using the developer tools of your browser. The fastest way to figure out the corresponding API call.

[FLavalliere]

@rahmnstein mind to share how you created / manged environments with taht?

[timcrall]

Do anyone know if there is an API for adding approvals to an environment? I cannot find it, maybe it does not exist yet.

https://docs.microsoft.com/en-us/rest/api/azure/devops/approvalsandchecks/check%20configurations/add?view=azure-devops-rest-6.1&viewFallbackFrom=azure-devops-rest-6.0

I think is the endpoint. But looks like you need to put in "resource":{"type":"environment","id":"your_environment_id","name":"your_env_name"}

that's based partially on using Developer Tools while creating one via the ui

[Ben10k]

@rahmnstein I just figured the approvals.
The API is documented here https://docs.microsoft.com/en-us/rest/api/azure/devops/approvalsandchecks/check%20configurations/add?view=azure-devops-rest-6.1

The resource.type must be set to environment and that's it.

[a2m1]

Might be not the cleanest but here is a working solution for env+approvals creation

1. Define environments with approvers

locals {
  environments = {
    "dev" = {}
    "qa" = {
      "approvers" = [
        {
          "displayName": azuredevops_group.group[<GROUP1>].display_name,
          "id": azuredevops_group.group[<GROUP1>].origin_id
        },
        {
          "displayName": azuredevops_group.group[<GROUP2>].display_name,
          "id": azuredevops_group.group[<GROUP2>].origin_id
        },
        {
          "displayName": azuredevops_group.group[<GROUP3>].display_name,
          "id": azuredevops_group.group[<GROUP3>].origin_id
        }
      ]
      minRequiredApprovers = 1
      timeout = 480
    }
  }
}

2. Create environments

resource "null_resource" "environments" {
  for_each = local.environments 
  triggers = {
    name                       = each.key
    description                = lookup(each.value, "description", "Created for approval")
    azdo_pat_token_owner_email = var.azdo_pat_token_owner_email
    azdo_pat_token             = var.azdo_pat_token
    project_url                = "${data.azuredevops_client_config.current.organization_url}/${azuredevops_project.project.name}"
  }
  provisioner "local-exec" {
    command = <<-EOT
      curl -X POST \
        -u "${self.triggers.azdo_pat_token_owner_email}:${self.triggers.azdo_pat_token}" \
        -H "Content-Type: application/json" \
        --data "{\"name\":\"${self.triggers.name}\",\"description\":\"${self.triggers.description}\"}" \
        "${self.triggers.project_url}/_apis/distributedtask/environments?api-version=6.0-preview.1"
    EOT
  }
}

3. Get IDs of created environments via external data

data "external" "environments" {
  program = ["./data_external_environment.py"]
  query = {
    azdo_pat_token_owner_email = var.azdo_pat_token_owner_email
    azdo_pat_token             = var.azdo_pat_token
    project_url                = "${data.azuredevops_client_config.current.organization_url}/${azuredevops_project.project.name}"
  }
}

Where data_external_environment.py

#!/usr/bin/env python3
import json
from os import environ
import sys
import requests


for linenum, line in enumerate(sys.stdin):
    if linenum == 0:
        data=json.loads(line)

response = requests.get(
    f'{data["project_url"]}/_apis/distributedtask/environments?api-version=6.0-preview.1',
    auth=(data["azdo_pat_token_owner_email"], data["azdo_pat_token"])
)

environments = response.json().get('value')
envs_dict = {}
for environment in environments:
    env_id = environment.get('id')
    env_name = environment.get('name')
    envs_dict[env_name] = str(env_id)

print(json.dumps(envs_dict))

4. Create Approval checks

resource "null_resource" "checks" {
  for_each = local.environments 
  triggers = {
    approvers                  = replace(jsonencode(lookup(each.value, "approvers", [])), "\"", "\\\"")
    minRequiredApprovers       = lookup(each.value, "minRequiredApprovers", 0)
    timeout                    = lookup(each.value, "timeout", 480)
    resource                   = replace(jsonencode({
      "type" = "environment",
      "id" = data.external.environments.result[each.key],
      "name" = each.key
    }), "\"", "\\\"")
    azdo_pat_token_owner_email = var.azdo_pat_token_owner_email
    azdo_pat_token             = var.azdo_pat_token
    project_url                = "${data.azuredevops_client_config.current.organization_url}/${azuredevops_project.project.name}"
  }
  provisioner "local-exec" {
    command = <<-EOT
      curl -s -X POST \
        -u "${self.triggers.azdo_pat_token_owner_email}:${self.triggers.azdo_pat_token}" \
        -H "Content-Type: application/json" \
        --data "{\"settings\":{\"approvers\":${self.triggers.approvers},\"executionOrder\":\"anyOrder\",\"minRequiredApprovers\":${self.triggers.minRequiredApprovers},\"instructions\":\"Instructions\",\"blockedApprovers\":[]},\"timeout\":${self.triggers.timeout},\"type\":{\"id\":\"8c6f20a7-a545-4486-9777-f762fafe0d4d\",\"name\":\"Approval\"},\"resource\":${self.triggers.resource}}" \
        "${self.triggers.project_url}/_apis/pipelines/checks/configurations?api-version=6.1-preview.1"
    EOT
  }
}

[aleqsss]

If you take a look at #204, you can see that @tmeckel mentions the merge of v6.0 of the Azure DevOps Services REST API to the azure-devops-go-api:

microsoft/azure-devops-go-api#110

I guess this is the first step towards implementing the functionality in this provider. 🙂

[tmeckel]

@aleqsss check my last comment at #204 It seems that there are structural issues with new v6 GO API

[jholm117]

Is this unblocked after #494 ?

[xuzhang3]

Is this unblocked after #494 ?

@jholm117 Yes, since we have upgrade the SDK from v5 -> v6, this is unblocked now

[lisaplapla]

@xuzhang3 we need to deploy environments using terraform too. Do you have an approximate ETA for this resource to be added to the provider and a new version released, or should we use one of the workarounds described in this thread?

[xuzhang3]

@lisaplapla This task is in our backlog but not the first priority, you can use the workaround in this thread.