Cross-Tenant subscription deployments

[itpropro]

Is your feature request related to a problem? Please describe.
We have multiple customer cases, where we need to deploy VMs based on ARM/Bicep templates with image references. These images are located in a Azure Compute Library in another tenant and have a specific, second service principal they use to authenticate.
The docs doesn't mention this scenario, only using CLI and PowerShell to realize cross-tenant access. Also, the parameters aux-tenants and aux-subs from az deployment group are not available in az deployment sub.

Describe the solution you'd like
We need a solution where we can specify a second service principal and tenant to authenticate against the compute gallery. The reference to the image is already in resource id format. This has to be provided either as a parameter in Bicep(maybe with scopes) or as a azure cli parameter.

Describe alternatives you've considered
Afaik there are no alternatives available, if you want to use IaC with ARM or Bicep

[yonzhan]

ARM

[ghost]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @armleads-azure.

Issue Details

---

Is your feature request related to a problem? Please describe.
We have multiple customer cases, where we need to deploy VMs based on ARM/Bicep templates with image references. These images are located in a Azure Compute Library in another tenant and have a specific, second service principal they use to authenticate.
The docs doesn't mention this scenario, only using CLI and PowerShell to realize cross-tenant access. Also, the parameters aux-tenants and aux-subs from az deployment group are not available in az deployment sub.

Describe the solution you'd like
We need a solution where we can specify a second service principal and tenant to authenticate against the compute gallery. The reference to the image is already in resource id format. This has to be provided either as a parameter in Bicep(maybe with scopes) or as a azure cli parameter.

Describe alternatives you've considered
Afaik there are no alternatives available, if you want to use IaC with ARM or Bicep

Author:	itpropro
Assignees:	zhoxing-ms
Labels:	Service Attention, ARM, customer-reported, feature-request, Auto-Assign
Milestone:	Backlog

[zhoxing-ms]

This feature request requires the ARM service team to confirm whether the REST create-or-update-at-subscription-scope can support the deployment across tenant by using x-ms-authorization-auxiliary like the REST create-or-update-at-group-scope
@alex-frankel Could you please answer this question?

[alex-frankel]

It looks like we do not support this header and we do not have any work scheduled at the moment to start supporting it.

[zhoxing-ms]

@alex-frankel Just curiosity, why only create-or-update-at-group-scope supports x-ms-authorization-auxiliary , but create-or-update-at-subscription-scope does not support x-ms-authorization-auxiliary , is this just because there is no such requirement and plan before?

[itpropro]

@alex-frankel what is the current way to deploy a bicep or arm template for a VM that is based on a Azure Compute Gallery image from another tenant? This is a common problem I have and I would really like to know what the Microsoft recommended way is, as the base recommendation is to use IaC wherever possible.

[alex-frankel]

@bmoore-msft -- do you happen to know anything about this? My understanding is that cross-tenant VM image access is possible in some capacity, but I do not know the details. It might also be helpful to reach out to someone on the compute team via a support case if we can't figure it out here.

[bmoore-msft]

@alex-frankel - are you sure we don't support the x-ms-authorization-auxiliary header in sub-level deployments? in order for it to work on RGs it would have to work across subs since an RG in a different tenant also has to be in a different sub... which would mean we had to disable something - which while odd, not unheard of...

@itpropro - have you tried sending the deployment via REST? It would mean you need to set the header yourself to try it... I appreciate that it may be too onerous to try, since I thought about how onerous it would be for me to go set up a multi-tenant repro. Another option you have here is to deploy the initial template at RG scope (which seems to support what you need).

[alex-frankel]

are you sure we don't support the x-ms-authorization-auxiliary header in sub-level deployments?

@bmoore-msft -- definitely not sure. This was what I was told when I spoke to Marc. Do you have any examples of using the header at RG scope? That'll be helpful in getting a definitive answer.

[bmoore-msft]

ok, I think the "gap" may be creating deployments x-tenant. You can set the header on the deployment and the header would (or should) be passed to the resources in that deployment. But the header isn't passed on nested deployments (this is true for RG scoped deployments).

So it depends on the resources being created in the sub-level deployment. If those needed (or supported) x-tenant scenarios it should work, but if you nested anything (like a cross-tenant-sub) deployment that is not supported.

@itpropro - interested in your scenario and how it fits here... if you're trying to deploy a VM and that needs access to an image gallery in another tenant - that should work in a RG scoped deployment (as long as it's not nested). Also assuming SIG supports this...

[itpropro]

@bmoore-msft unfortunately it doesn't work and regarding to the docs, the only possibility is to use PowerShell or Azure CLI as all other deployment methods are not able to handle two different credential pairs at the same time. The problem is not access or the different tenant IDs, but the fact that you normally need two different sets of service principal credentials:

1. The service principal that has access to the image gallery and images in the source tenant
2. The service principal that is used for the actual (vm) deployment in the target tenant

[bmoore-msft]

@itpropro - can you be more specific about what "doesn't work"? Here's my understanding from the thread:

1. cross-tenant authorization for subscription level deployments is not supported in Azure... so this is not a CLI issue but a platform one (CLI can't fix this)
2. in the OP the scenario called out is:

We need a solution where we can specify a second service principal and tenant to authenticate against the compute gallery. The reference to the image is already in resource id format. This has to be provided either as a parameter in Bicep(maybe with scopes) or as a azure cli parameter.

To create a VM with an image from a SIG in another tenant, you need a resourceGroup level deployment. This can be done today using bicep and the CLI using the --aux-tenants parameter for az group deployment create. I don't think PS supports this (or portal) but any REST caller will. E.g. in PowerShell Invoke-RestMethod would support it (since auth headers are manually set).

So #1 is blocked for CLI by platform limitations, #2 is currently supported. Is there another scenario I'm missing?

[itpropro]

Hi @bmoore-msft, the scenario that doesn't work is the one described in the OP for automation. If I have not missed anything, it's not possible to specify credentials for automated deployment with the --aux-tenants parameter, so it's not usable for automation.
If the answer for this scenario is that it is just not possible with Azure to do an automated (non-interactive) deployment of vm with images from a compute gallery located in another tenant, that's totally fine for me, the customer will probably switch to terraform then or find another non-native solution.

[bmoore-msft]

The aux-tenants param takes a tenantId (not creds) - CLI will go fetch the token from that tenant for the current user. So basically you have something like:

az deployment group create -g myRG -f .\vm.json -p "@parameters.json" --aux-tenants "6457d1f2-4394-4fc2-b163-e46ffcbbec5c"

In the REST api you have to supply the token, but CLI makes that much nicer by fetching the token for you if you just give it the tenantId... That does work today.

Does that help or am I still off somewhere?

[itpropro]

Thanks for the reply @bmoore-msft, the problem is that the current user is not the one with permissions in both tenants. There are two different service principals, one for Tenant 1 and one for Tenant 2. So --aux-tenants doesn't really help in that automation case, as I cannot provide 2 different service principals with it.

[bmoore-msft]

Ok, I'm with you... that's not supported, all the tokens need to be from the same principal - see: https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/authenticate-multi-tenant#header-values-for-authentication

[itpropro]

I think I have missed to state that clearer, but that was what I meant in the issue text saying "We need a solution where we can specify a second service principal and tenant to authenticate against the compute gallery." 😅
Do you have any idea if there is a solution for this on the horizon?

[bmoore-msft]

Not on the horizon that I'm aware of - the design is intentional due to the surface area it creates.