Feature: cloudflare_tunnel_config resource

[jtcressy]

Current Terraform and Cloudflare provider version

latest as of writing this issue

Description

This resource should accompany the existing cloudflare_argo_tunnel resource (or cloudflare_tunnel once renamed by #1646)

The new cfd_tunnel api supported by cloudflare-go's Tunnel api includes the all-new Tunnel Configuration API now implemented by cloudflare-go#948

Use cases

Goal: In this resource, terraform should be able to configure the new cloudflared tunnels using the latest remote configuration API

Potential Terraform configuration

resource "cloudflare_argo_tunnel" "tunnel" {
  account_id = "c4a7362d577a6c3019a474fd6f485821"
  name       = "my_tunnel"
  secret     = "AQIDBAUGBwgBAgMEBQYHCAECAwQFBgcIAQIDBAUGBwg="
}

resource "cloudflare_tunnel_config" "tunnel" {
  account_id         = "c4a7362d577a6c3019a474fd6f485821"
  tunnel_id          = cloudflare_argo_tunnel.tunnel.id

  config { // Optional, only one instance of config allowed
    warp_routing { // Optional, only one instance of warp_routing allowed
      enabled = true
    }
    origin_request { // Optional, only one instance of origin_request allowed
      connect_timeout = "60s"
      tls_timeout = "60s"
      tcp_keep_alive = "60s"
      no_happy_eyeballs = false
      keep_alive_connections = 1024
      keep_alive_timeout = "60s"
      http_host_header = "baz"
      origin_server_name = "foobar"
      ca_pool = "/path/to/unsigned/ca/pool"
      no_tls_verify = false
      disable_chunked_encoding = false
      bastion_mode = false
      proxy_address = "127.0.0.1"
      proxy_port = "8123"
      proxy_type = "socks"
      ip_rule { // Can be repeated multiple times to define more ip rules for proxy service
        prefix = "/web"
        ports = [80, 443]
        allow = false
      }
    }
    ingress_rule { // Can be repeated multiple times to define more ingress rules
      hostname = "foo"
      path = "/bar"
      service = "127.0.0.1:8080"
    }
  }
}

References

• https://github.com/cloudflare/cloudflare-go/blob/master/tunnel.go
• Add UpdateTunnelConfiguration func cloudflare-go#948
• https://api.cloudflare.com/#cloudflare-tunnel-configuration-put-configuration

[jacobbednarz]

cloudflare_argo_tunnel is already using the newer APIs and can be referenced for other resources such as tunnel_route and tunnel_virtual_network. this resource will be renamed in 4.x - see #1646.

at this stage, i'm not sure if we'll put the configuration properties on this resource or a new resource all together as they are separate endpoints.

i'd recommend updating this issue to purely track the addition of the configuration resource as the other points are already covered by other issues.

[jtcressy]

cloudflare_argo_tunnel is already using the newer APIs and can be referenced for other resources such as tunnel_route and tunnel_virtual_network. this resource will be renamed in 4.x - see #1646.

at this stage, i'm not sure if we'll put the configuration properties on this resource or a new resource all together as they are separate endpoints.

i'd recommend updating this issue to purely track the addition of the configuration resource as the other points are already covered by other issues.

I agree with making this a separate resource. I also didn't realize #1646 had plans to rename the resource.

Also, while cloudflare-go updated the argo tunnel functions to the new API endpoint, it may still be in our best interest to update the function references here to point at the new tunnel functions and avoid library-level deprecations: https://github.com/cloudflare/terraform-provider-cloudflare/blob/master/internal/provider/resource_cloudflare_argo_tunnel.go#L35
e.g. client.CreateArgoTunnel -> client.CreateTunnel

[hatemosphere]

It gets quite confusing when i try to use cloudflare_argo_tunnel resource for managed tunnel and routes for it. So right after tunnel creation i see this in my Zero Trust Dashboard:

And the tunnel is actually NOT managed, cloudflared tunnel connectors installed with cloudflared service install "${tunnel_token}" just confirming this:
2022-07-09T07:19:59Z ERR cloudflared received a request from WARP client, but your configuration has disabled ingress from WARP clients. To enable this, set "warp-routing:\n\t enabled: true" in your config.yaml

Right after manual migration to a managed tunnel via UI everything starts working, including private network routes previously configured via Terraform. Also, the same "local" tunnel gets created when i try to use "new" API directly: https://api.cloudflare.com/#cloudflare-tunnel-create-cloudflare-tunnel
So maybe it's an issue with the API itself, not allowing to create "remotely managed" tunnels directly, please let me know if i should create a separate issue somewhere else...

TL;DR - cloudflare_argo_tunnel makes no sense (or maybe i'm doing something wrong) when it comes to configuration of remotely managed Zero Trust private tunnels, you still have to migrate them via UI, even considering that it allows adding networks with cloudflare_tunnel_virtual_network and routes for them via cloudflare_tunnel_route, and even says Remote Configuration for Cloudflare tunnel here: https://api.cloudflare.com/#cloudflare-tunnel-configuration-properties

[jtcressy]

@hatemosphere This is exactly what i'm hoping to get solved with this resource. Cloudflare tunnels are not "remote managed" until configuration is PUT at accounts/:account_identifier/cfd_tunnel/:tunnel_id/configurations

Currently, the cloudflare_argo_tunnel resource does not put anything at this api, and the proposed cloudflare_tunnel_config resource by this issue aims to cover this api endpoint.

[christidis]

I am evaluating Terraform managed Cloudflare Tunnels for exposing Kubernetes workloads.

It would be really nice to be able to create Argo "Zero Trust" tunnels (managed by Cloudflare) and configure "Public Hostnames" and "Private Networks" in them and manage everything with Terraform. This will eliminate the need of having to define Public Hostnames and their mapping with Kubernetes services in a Helm Chart outside Terraform, as everything will be configured in one place.

Subscribing for future updates.

[Nicarim]

Is there any update on when this will be implemented? I'm struggling to figure out how to add public hostname via terraform same way it's available through UI if you change the tunnel to dashboard managed mode. It seems there is no way to do it currently through this terraform provider, is that correct?

[edjackson-wf]

cloudflare_argo_tunnel is already using the newer APIs and can be referenced for other resources such as tunnel_route and tunnel_virtual_network. this resource will be renamed in 4.x - see #1646.

I assume this is referring to the accounts/:account_identifier/cfd_tunnel/:tunnel_id/ endpoints. If the provider has been updated to use these, shouldn't it also support the full CRUD capabilities they provide? Last I checked, the existing cloudflare_argo_tunnel resource still has no update capability. I suppose this made sense when all tunnels were locally managed, but now I'd like to be able to rotate the tunnel secret via terraform. I think this goes hand in hand with the configuration resource (since applying that is what converts the tunnel to be remotely managed).

I've been doing some unpleasant workarounds for these issues by using the mastercard/restapi provider, and would be very pleased to see full support for the cfd_tunnel and its configuration in the TF provider as soon as possible.

[mmllc-jsilverman]

As a related aside, it would be nice if, at a minimum, the documentation would be updated to reflect the manual migration step, and the fact that one must either do this manual migration in the UI or find a way to automate the provisioning of a cloudflared config.yml with ingress rules.

Non-inclusive list of documentation locations that should mention this:

• https://registry.terraform.io/providers/cloudflare/cloudflare/latest/docs/resources/tunnel_route
• https://developers.cloudflare.com/cloudflare-one/connections/connect-apps/install-and-setup/tunnel-guide/local/local-management/configuration-file/

A clearer description in the docs would have saved me -- and I'm sure someone else -- a lot of headache.

[elev8studio]

Is there any update on when we will be able to set the public hostnames for a cloudflare tunnel using terraform?

In the meantime, I'm struggling to know if there's another way to do this. Currently does it have to be done manually in the ZeroTrust dashboard? Or can I use a config file on the server to manage the public hostnames for a tunnel?

A bit of context, I'm running cloudflare/cloudflared as a service in a docker-compose.yml file. I would like to know (in the meantime) how to configure the public hostnames programatically.

Help appreciated.

[mmllc-jsilverman]

@elev8studio -- you can set the public hostnames using Terraform. You just can't configure the tunnel client as to what you want the hostnames to do. (i.e. the Ingress rules in /etc/cloudflared/config.yml)

I automated the process of creating tunnels by

• Terraform/cloudflare provider to create the tunnel itself
• Terraform/Cloudflare provider to create CNAME from tunnel ID created in first step
• shell script to automate installation of cloudflared on host
• shell script to create the /etc/cloudflared/config.yml file to configure the running cloudflared client <-- This is the step that should be possible via the Terraform provider but is not

[elev8studio]

@mmllc-jsilverman Yep, you're right. It's the last step that I want to be able to do with Terraform. Hopefully they'll make it possible before too long. Would you mind sharing the shell scripts in a gist with the details of your last 2 steps? I want to automate as much as possible. I'm also running cloudflared as a docker container with docker compose. Any support you can give would be greatly appreciated.

[mmllc-jsilverman]

@elev8studio sure...

https://gist.github.com/mmllc-jsilverman/2c8a363134d10e55fdccda27ec68dcd8

You will obviously have to change some things to suit your environment

[elev8studio]

@mmllc-jsilverman Thanks very much!

Please can you explain how you use this script in your automation flow? It might help me determine my course of action.

Please can you clarify the mapping of the variables in your script to the Terraform output, specifically the cf_account_tag and cf_tunnel_secret? I'm not seeing all of them here, and I'm confused between the base64 encoded secret and tunnel_token output—which one is required in the config.yml file to run cloudflared?

Thanks for your help!

[mmllc-jsilverman]

The script is part of the provisioner for the VM on which it runs.

For docker, it would be a series of RUN statements in the Dockerfile.

As far as those variables, there are a variety of ways to inject variables into an automation workflow. Some of the values are things that have to be provided on instantiation. I used Terraform templatefile() and injected the script into the GCP metadata startup. https://developer.hashicorp.com/terraform/language/functions/templatefile

[github-actions]

This functionality has been released in v3.29.0 of the Terraform Cloudflare Provider.

Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

[christidis]

Thanks for this. Can you use origin_request inside an ingress_rule for ingress specific overwrites? I tried it and it failed.

like for example supporting the following origin configuration for localhost:8002
https://developers.cloudflare.com/cloudflare-one/connections/connect-apps/install-and-setup/tunnel-guide/local/local-management/ingress/#origin-configuration

tunnel: 6ff42ae2-765d-4adf-8112-31c55c1551ef
credentials-file: /root/.cloudflared/6ff42ae2-765d-4adf-8112-31c55c1551ef.json
originRequest: # Top-level configuration
  connectTimeout: 30s

ingress:
  # The localhost:8000 service inherits all root-level configuration.
  # In other words, it will use a connectTimeout of 30 seconds.
  - hostname: example.com
    service: localhost:8000
  - hostname: example2.com
    service: localhost:8001
  # The localhost:8002 service overrides some root-level config.
  - service: localhost:8002
    originRequest:
      connectTimeout: 10s
      disableChunkedEncoding: true
  # Some built-in services such as `http_status` do not use any configuration.
  # The service below will simply respond with HTTP 404.
  - service: http_status:404

Do I need to maintain a cloudflare_tunnel_config for each ingress? kind of confused

CC @Cyb3r-Jak3

[Cyb3r-Jak3]

So it does not look like the API offers a (public) way to set originRequest on an ingress rule. This PR, cloudflare/cloudflare-go#1099, will add the support that is needed. The only thing that is needed first is the public API updated. I would recommend making a new issue in this repo to track this feature.

[christidis]

Thanks @Cyb3r-Jak3. Makes sense. I have opened issues/2072.