AzureRM Provider 2.34 fails to authenticate on US Government (Regression from 2.33)

[vsabella]

In the latest azurerm provider 2.34, authenticating via Azure CLI against Azure Us Government no longer works:

This does still work in azurerm provider 2.33, it is a regression in 2.34

Terraform (and AzureRM Provider) Version

az version
{
  "azure-cli": "2.14.0",
  "azure-cli-core": "2.14.0",
  "azure-cli-telemetry": "1.0.6",
  "extensions": {}
}

Terraform v0.13.5

Linux my-ubuntu-20.04 5.4.0-52-generic #57-Ubuntu SMP Thu Oct 15 10:57:00 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Expected Result

terraform plan authorizes.

Actual Behavior

Error: Error obtaining Authorization Token from the Azure CLI: Error parsing json result from the Azure CLI: Error waiting for the Azure CLI: exit status 1

  on providers.tf line 3, in provider "azurerm":
   3: provider "azurerm" {

Steps to Reproduce

Initializing provider plugins...
- Using previously-installed hashicorp/azurerm v2.34.0

variable "cloud" {}

provider "azurerm" {
  version = "~> 2.34.0"
  tenant_id       = var.cloud.azure.tenant_id
  subscription_id = var.cloud.azure.subscription_id
  environment     = var.cloud.azure.environment
  features {}
}

resource "azurerm_resource_group" "foobar" {
  name                = "foobar"
  location            = "USGov Virginia"
}

Debug Output

*** Azure CLI Log ***

_id":"****<masked>****",
"error_uri":"https://login.microsoftonline.us/error?code=500011"}

ERROR : cli.azure.cli.core.azclierror : 
Get Token request returned http error: 400 and server response: 

{"error":"invalid_resource",
"error_description":"AADSTS500011: The resource principal named N/A was not found in the tenant named ****<masked>****.

This can happen if the application has not been installed by the administrator of the tenant or consented to 
by any user in the tenant. 

{"error":"invalid_resource",
"error_description":"AADSTS500011: The resource principal named N/A was not found in the tenant named ***<masked>***. 

This can happen if the application has not been installed by the administrator of the tenant or consented to 
by any user in the tenant. You might have sent your authentication request to the wrong tenant.

Trace ID:****<masked>****
Correlation ID: ****<masked>****
error_codes:[500011],
error_uri":"https://login.microsoftonline.us/error?code=500011"

Traceback (most recent call last):
  File "/opt/az/lib/python3.6/site-packages/knack/cli.py", line 215, in invoke
    cmd_result = self.invocation.execute(args)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 654, in execute
    raise ex
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 718, in _run_jobs_serially
    results.append(self._run_job(expanded_arg, cmd_copy))
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 711, in _run_job
    six.reraise(*sys.exc_info())
  File "/opt/az/lib/python3.6/site-packages/six.py", line 703, in reraise
    raise value
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 688, in _run_job
    result = cmd_copy(params)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 325, in __call__
    return self.handler(*args, **kwargs)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/__init__.py", line 784, in default_command_handler
    return op(**command_args)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/command_modules/profile/custom.py", line 75, in get_access_token
    creds, subscription, tenant = profile.get_raw_token(subscription=subscription, resource=resource, tenant=tenant)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/_profile.py", line 650, in get_raw_token
    tenant_dest, resource)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/_profile.py", line 1019, in retrieve_token_for_user
    token_entry = context.acquire_token(resource, username, _CLIENT_ID)
  File "/opt/az/lib/python3.6/site-packages/adal/authentication_context.py", line 145, in acquire_token
    return self._acquire_token(token_func)
  File "/opt/az/lib/python3.6/site-packages/adal/authentication_context.py", line 128, in _acquire_token
    return token_func(self)
  File "/opt/az/lib/python3.6/site-packages/adal/authentication_context.py", line 143, in token_func
    return token_request.get_token_from_cache_with_refresh(user_id)
  File "/opt/az/lib/python3.6/site-packages/adal/token_request.py", line 347, in get_token_from_cache_with_refresh
    return self._find_token_from_cache()
  File "/opt/az/lib/python3.6/site-packages/adal/token_request.py", line 127, in _find_token_from_cache
    return self._cache_driver.find(cache_query)
  File "/opt/az/lib/python3.6/site-packages/adal/cache_driver.py", line 199, in find
    is_resource_tenant_specific)
  File "/opt/az/lib/python3.6/site-packages/adal/cache_driver.py", line 184, in _refresh_entry_if_necessary
    return self._acquire_new_token_from_mrrt(entry)
  File "/opt/az/lib/python3.6/site-packages/adal/cache_driver.py", line 160, in _acquire_new_token_from_mrrt
    token_response = self._refresh_function(entry, self._resource)
  File "/opt/az/lib/python3.6/site-packages/adal/token_request.py", line 137, in _get_token_with_token_response
    return self._get_token_with_refresh_token(refresh_token, resource, None)
  File "/opt/az/lib/python3.6/site-packages/adal/token_request.py", line 339, in _get_token_with_refresh_token
    return self._oauth_get_token(oauth_parameters)
  File "/opt/az/lib/python3.6/site-packages/adal/token_request.py", line 112, in _oauth_get_token
    return client.get_token(oauth_parameters)
  File "/opt/az/lib/python3.6/site-packages/adal/oauth2_client.py", line 289, in get_token
    raise AdalError(return_error_string, error_response)
adal.adal_error.AdalError: Get Token request returned http error: 400 and server response: 

Important Factoids

Previously Working Example

This results in a proper plan. Can be reproduced 100% of the time.

Initializing provider plugins...
- Finding hashicorp/azurerm versions matching "~> 2.33.0"...
- Installing hashicorp/azurerm v2.33.0...
- Installed hashicorp/azurerm v2.33.0 (signed by HashiCorp)

variable "cloud" {}

provider "azurerm" {
  version = "~> 2.33.0"
  tenant_id       = var.cloud.azure.tenant_id
  subscription_id = var.cloud.azure.subscription_id
  environment     = var.cloud.azure.environment
  features {}
}

resource "azurerm_resource_group" "foobar" {
  name                = "foobar"
  location            = "USGov Virginia"
}

[vsabella]

I tracked this down:

https://github.com/Azure/go-autorest/blob/master/autorest/azure/environments.go

sets all unavailable resources to "N/A"

However when Synapse tokens were added to the provider, it attempts to create a client with "N/A" as the endpoint, which the hashicorp go-helpers happily pass through to az account get-access

https://github.com/terraform-providers/terraform-provider-azurerm/blob/2c7ae3cfcc59aad45239d021aa64198dd1701d51/azurerm/internal/clients/builder.go#L100

It seems like these should be defaulted to "" or a more formal "N/A" model like null or optional makes sense than just a sentinel string. Or at least lazy-provision the tokens.

[njuCZ]

@vsabella sorry for causing this regression issue for other clouds. I have submit a quick PR to fix it.

[TheMacStack]

Just to mention this also effects Azure China (Incase you were not already aware)

[vsabella]

@vsabella sorry for causing this regression issue for other clouds. I have submit a quick PR to fix it.

No worries, I appreciate the fast response. For now we just pinned to 2.33 no problem.

[ghost]

This has been released in version 2.35.0 of the provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading. As an example:

provider "azurerm" {
    version = "~> 2.35.0"
}
# ... other configuration ...

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 [email protected]. Thanks!