Unable to update existing virtual machine with managed disks

[dpedu2]

Terraform Version

Terraform v0.11.2

Affected Resource(s)

• azurerm_virtual_machine

Terraform Configuration Files

I have an azure vm resource that's fairly similar to the documented example:

resource "azurerm_virtual_machine" "test" {
    name                  = "acctvm"
    location              = "${azurerm_resource_group.clusterrg.location}"
    resource_group_name   = "${azurerm_resource_group.clusterrg.name}"
    network_interface_ids = ["${azurerm_network_interface.vmiface.id}"]
    vm_size               = "Standard_DS1_v2"
    delete_os_disk_on_termination = false
    delete_data_disks_on_termination = false

    availability_set_id = "${azurerm_availability_set.test.id}"

    storage_image_reference {
        publisher = "Canonical"
        offer     = "UbuntuServer"
        sku       = "16.04-LTS"
        version   = "latest"
    }

    storage_os_disk {
        name              = "myosdisk1"
        caching           = "ReadWrite"
        create_option     = "FromImage"
        managed_disk_type = "Standard_LRS"
    }

    storage_data_disk {
        name            = "${azurerm_managed_disk.test.name}"
        managed_disk_id = "${azurerm_managed_disk.test.id}"
        create_option   = "Attach"
        lun             = 1
        disk_size_gb    = "${azurerm_managed_disk.test.disk_size_gb}"
    }

    os_profile {
      computer_name  = "testvm"
      admin_username = "ubuntu"
    }

    os_profile_linux_config {
        disable_password_authentication = true
        ssh_keys {
            path     = "/home/ubuntu/.ssh/authorized_keys"
            key_data = "ssh-rsa <snip>"
        }
    }
}

Some time later, I want to add another data disk, so I add a resource for my new disk:

resource "azurerm_managed_disk" "test2" {
  name                 = "datadisk_existing2"
  location             = "${azurerm_resource_group.clusterrg.location}"
  resource_group_name  = "${azurerm_resource_group.clusterrg.name}"
  storage_account_type = "Standard_LRS"
  create_option        = "Empty"
  disk_size_gb         = "1023"
}

And add a block like this within the azurerm_virtual_machine block:

    storage_data_disk {
        name            = "${azurerm_managed_disk.test2.name}"
        managed_disk_id = "${azurerm_managed_disk.test2.id}"
        create_option   = "Attach"
        lun             = 2
        disk_size_gb    = "${azurerm_managed_disk.test2.disk_size_gb}"
    }

Things look good when I plan (unchanged attributes snipped out):

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create
-/+ destroy and then create replacement

Terraform will perform the following actions:

  + azurerm_managed_disk.test2
      id:                                                               <computed>
      create_option:                                                    "Empty"
      disk_size_gb:                                                     "1023"
      location:                                                         "westus"
      name:                                                             "datadisk_existing2"
      resource_group_name:                                              "davetest2"
      source_uri:                                                       <computed>
      storage_account_type:                                             "Standard_LRS"
      tags.%:                                                           <computed>

-/+ azurerm_virtual_machine.test (new resource required)
      id:                                                               "/subscriptions/<snip>/resourceGroups/davetest2/providers/Microsoft.Compute/virtualMachines/acctvm" => <computed> (forces new resource)
      ...
      os_profile.2545405607.admin_password:                             <sensitive> => <sensitive> (attribute changed)
      ...
      storage_data_disk.1.managed_disk_id:                              "" => "${azurerm_managed_disk.test2.id}" (forces new resource)


Plan: 2 to add, 0 to change, 1 to destroy.

However, when I execute the plan, things get a little weird. The new data disk is created and the original VM destroyed as expected. However, it hits an error trying to create the VM again:

Error: Error applying plan:

1 error(s) occurred:

* azurerm_virtual_machine.test: 1 error(s) occurred:

* azurerm_virtual_machine.test: compute.VirtualMachinesClient#CreateOrUpdate: Failure sending request: StatusCode=200 -- Original Error: Long running operation terminated with status 'Failed': Code="ConflictingUserInput" Message="Disk myosdisk1 already exists in resource group DAVETEST2. Only CreateOption.Attach is supported."

Which seems to indicate I should change the create_option of storage_os_disk to Attach instead of FromImage, which sort of makes sense since the disk already exists and I want it attached to a new VM.

But when create_option is set to FromImage, even after plan tells me the new configuration is OK, there's another error when I try to run the plan:

Error: Error applying plan:

1 error(s) occurred:

* azurerm_virtual_machine.test: 1 error(s) occurred:

* azurerm_virtual_machine.test: [ERROR] Must specify `vhd_uri` or `managed_disk_id` to attach

Why do I need to specify this? Where do I even get this? Terraform handled creating and attaching the disk for me, why do I need to be this explicit to make a simple change on an unrelated property of the VM?

Can I even specify this?

If I try to add vhd_uri or managed_disk_id, I also need to delete managed_disk_type = "Standard_LRS". I tried adding a managed disk data source, which is documented as providing a "source_uri" attribute, but when I tried to use it I get:

Error: Error running plan: 1 error(s) occurred:

* azurerm_virtual_machine.test: 1 error(s) occurred:

* azurerm_virtual_machine.test: Resource 'data.azurerm_managed_disk.theosdisk' does not have attribute 'source_uri' for variable 'data.azurerm_managed_disk.theosdisk.source_uri'

Since it's a managed disk, maybe that's expected? Either way, unclear.

In the example for the azurerm_managed_disk above, use of the id attribute is shown but the attribute isn't documented. I'm assuming this is the same as the managed_disk_id param the VM's storage_os_disk section wants, there's another error, again not caught by planning:

Error: Error applying plan:

1 error(s) occurred:

* azurerm_virtual_machine.test: 1 error(s) occurred:

* azurerm_virtual_machine.test: compute.VirtualMachinesClient#CreateOrUpdate: Failure sending request: StatusCode=409 -- Original Error: failed request: autorest/azure: Service returned an error. Status=<nil> Code="PropertyChangeNotAllowed" Message="Changing property 'osDisk.createOption' is not allowed."

I don't understand this either, it already deleted the machine and I want to create a new one, not change any properties.

At this point I'm completely stuck. Terraform / azurerm seems unable to recreate the machine from existing disks. The only way I've been able to get back in a good state is to:

1. Via Azure's UI, manually delete the virtual machine
2. Add os_type = "Linux" under storage_os_disk
3. Delete the storage_image_reference block under the VM
4. Delete the os_profile block under the VM
5. Plan then apply again

Am I missing something?

Debug Output

Plan apply: https://gist.github.com/dpedu2/4acc2dccd5d907053c823d9c891158ba

Expected Behavior

I should be able to create and modify Azure virtual machines

Actual Behavior

Updating azure virtual machines requires manual intervention and drastic altering of my terraform config.

Steps to Reproduce

See above

Important Factoids

I don't think anything I'm doing is particularly unique; I'm literally taking the "with managed disks" example VM resource, and, after initial creation, adding an additional virtual disk.

[jzampieron]

I think this is something I ran into a while back. It's an issue with the "FromImage" OS disk.

It wasn't deleted and terraform doesn't understand how to reattach it. AFAIK it's not actually terraform's fault, the Azure API itself lacks a "FromImageOrAttachIfExisting" option.

[DonEstefan]

I think @jzampieron means this issue: #243
As a workaround you can try to separate disk creation and vm creation.

change your code from

resource "azurerm_virtual_machine" "test" {
    storage_os_disk {
        create_option     = "FromImage"
    }
}

into something like

resource "azurerm_managed_disk" "osdisk" {
  create_option         = "FromImage"
}

resource "azurerm_virtual_machine" "test" {
    delete_os_disk_on_termination = false
    storage_os_disk {
        create_option      = "attach"
        managed_disk_id = "${azurerm_managed_disk.osdisk.id}"
    }
}

This should work as long as you use os-images from the marketplace. If you want to use self-made os-images you'll have to wait for #565

[clangaxon]

This is a more general issue that we have run into as well.

When you create any managed disks inline with the VM definition, you can destroy the VM leaving the disks and Terraform can't handle it because it's lost the disk state along with the VM definition.

I suspect, as mentioned above, that defining the disks outside of the VM will fix the problem, but we didn't do that as it was excessively verbose with no apparent benefit until #795 is implemented. Seeing #565 makes me suspect we have no workable solution here.

Even with a method to attach varying counts of disks, it seems like this method of managed disk handling needs to be fixed, removed, or heavily documented so that other people don't find themselves in this frustrating situation.

[dpedu2]

@DonEstefan's workaround of this issue works for me, thanks! I had tried similar but I'm not quite sure what I was doing wrong previously.

[VaijanathB]

I am unable to reproduce this issue in the current latest build. This issue is fixed as part of #813 which was released as part of 1.1.2 on Feb 15. As part of PR 813, we no longer recreate VM when we attach data disks. If you still face this issue, please reopen it with sample HCL scripts.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 [email protected]. Thanks!