IPv4 address issue with vSphere provider

[getstek]

Hi,

In the latest version of Terraform .69 using the vSphere provider I am facing an issue with NIC creation. The terraform.tf file looks like this:

# Enter the details of the vCenter here
variable "vcenter_address" {default = "x.x.x.x}
variable "vcenter_user" {default = "xxx"}
variable "vcenter_password" {default = "xxxxxx"}

# This is the prefix assigned to the machine
variable "short_name" {default = "dc01"}

# Some vSphere global variables
variable "vsphere_dc" {default = "LAB"}
variable "vsphere_cluster" {default = "LAB"}
variable "timezone" {default = "Asia/Dubai"}
variable "search_domain" {default = "watteel.local"}
variable "dns_servers" {default = "192.168.111.1,8.8.8.8"}

# Details for the master nodes
variable "master_cpu" {default = 2}
variable "master_mem" {default = 1024}
variable "master_count" {default = 1}
variable "master_template" {default = "Templates/CentOS7_x86_64"}
variable "master_datastore" {default = "ISCSI_HDD_LUN3_THIN"}
variable "master_network_label" {default = "VLAN111"}
variable "master_gateway" {default = "192.168.111.1"}
variable "master_netmask" {default = "24"}

provider "vsphere" {
  vsphere_server = "${var.vcenter_address}"
  user = "${var.vcenter_user}"
  password = "${var.vcenter_password}"
  allow_unverified_ssl = "True"
}

resource "vsphere_virtual_machine" "masters" {
    name = "${var.short_name}-control-${format("%02d", count.index+1)}"
    datacenter = "${var.vsphere_dc}"
    cluster = "${var.vsphere_cluster}"
    vcpu = "${var.master_cpu}"
    memory = "${var.master_mem}"
    disk {
        datastore = "${var.master_datastore}"
        template = "${var.master_template}"
    }
    gateway = "${var.master_gateway}"
    network_interface {
        label = "${var.master_network_label}"
        ipv4_address = "${var.master_ip_range}${format(count.index+1)}"
        ipv4_prefix_length = "${var.master_netmask}"
    }

  domain = "${var.search_domain}"
  dns_suffixes = ["${var.search_domain}"]
  dns_servers = ["${split(",", var.dns_servers)}"]
  time_zone = "${var.timezone}"
  count = "${var.master_count}"
}

The job runs, however no IPv4 interface is assigned. An IPv6 address appears and that's all.
From terraform.tfstate:

 "resources": {
                "vsphere_virtual_machine.masters": {
                    "type": "vsphere_virtual_machine",
                    "primary": {
                        "id": "dc01-control-01",
                        "attributes": {
                            "cluster": "LAB",
                            "datacenter": "LAB",
                            "disk.#": "1",
                            "disk.0.datastore": "ISCSI_HDD_LUN3_THIN",
                            "disk.0.iops": "0",
                            "disk.0.size": "0",
                            "disk.0.template": "Templates/CentOS7_x86_64",
                            "dns_servers.#": "2",
                            "dns_servers.0": "192.168.111.1",
                            "dns_servers.1": "8.8.8.8",
                            "dns_suffixes.#": "1",
                            "dns_suffixes.0": "watteel.local",
                            "domain": "watteel.local",
                            "gateway": "192.168.111.1",
                            "id": "dc01-control-01",
                            "memory": "1024",
                            "name": "dc01-control-01",
                            "network_interface.#": "1",
                            "network_interface.0.adapter_type": "",
                            "network_interface.0.ip_address": "",
                            "network_interface.0.ipv4_address": "",
                            "network_interface.0.ipv4_prefix_length": "0",
                            "network_interface.0.ipv6_address": "fe80::250:56ff:fe80:2c9a",
                            "network_interface.0.ipv6_prefix_length": "64",
                            "network_interface.0.label": "VLAN111",
                            "network_interface.0.subnet_mask": "",
                            "time_zone": "Asia/Dubai",
                            "vcpu": "2"
                        }

I've tried using both the old ip_address and subnet_mask fields from previous versions, as well as the new ipv4_address and prefix length.

Interestingly Terraform shows the correct IP on stdout when I run it.

[root@hatchery terraform]# terraform apply vsphere/
vsphere_virtual_machine.masters: Creating...
  cluster:                                "" => "LAB"
  datacenter:                             "" => "LAB"
  disk.#:                                 "" => "1"
  disk.0.datastore:                       "" => "ISCSI_HDD_LUN3_THIN"
  disk.0.template:                        "" => "Templates/CentOS7_x86_64"
  dns_servers.#:                          "" => "2"
  dns_servers.0:                          "" => "192.168.111.1"
  dns_servers.1:                          "" => "8.8.8.8"
  dns_suffixes.#:                         "" => "1"
  dns_suffixes.0:                         "" => "watteel.local"
  domain:                                 "" => "watteel.local"
  gateway:                                "" => "192.168.111.1"
  memory:                                 "" => "1024"
  name:                                   "" => "dc01-control-01"
  network_interface.#:                    "" => "1"
  network_interface.0.ip_address:         "" => "<computed>"
  network_interface.0.ipv4_address:       "" => "192.168.111.201"
  network_interface.0.ipv4_prefix_length: "" => "24"
  network_interface.0.ipv6_address:       "" => "<computed>"
  network_interface.0.ipv6_prefix_length: "" => "<computed>"
  network_interface.0.label:              "" => "VLAN111"
  network_interface.0.subnet_mask:        "" => "<computed>"
  time_zone:                              "" => "Asia/Dubai"
  vcpu:                                   "" => "2"
vsphere_virtual_machine.masters: Creation complete

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

Software versions:
vCenter 6.0 build 2559268
Terraform 0.6.9
Template: latest Centos 7 with open-vm-tools

[lesaux]

I am seeing the same behaviour. In fact I don't even have the network_interface.0.* fields in the tfstate file at all, yet everything looks fine on stdout.

[benlangfeld]

@lesaux I'm also seeing the network_interface* fields missing from the state file on the first apply run. They are then created by a plan run > 1 minute later. From debug logs, it appears the vSphere API is reporting no interfaces when the apply run refreshes state immediately after creating the VMs. I guess some part of the build process is asynchronous and Terraform isn't properly waiting for it to complete before reading the state of the VM.

Can you confirm this behaviour?

[benlangfeld]

I filed #6174 for my problem, @lesaux

[chrislovecnm]

@benlangfeld this may be related to #4283 ... Which is needs more testing. Let me know if you want to test a branch that I have.

[knuckolls]

I can confirm that I am also seeing this problem on v0.6.15

[chrislovecnm]

@knuckolls this problem is being worked on. We need to bump the govmomi vsphere API version, and are testing it.

[knuckolls]

@chrislovecnm i will happily test and perhaps contribute to whatever branch has that work on it

[chrislovecnm]

@knuckolls any and all help is appreciated. This is the PR that we are testing right now: #6479 ... To be clear it is only an API bump change. It does not have the "Wait for all IP's" that we are hoping will fix some of our network issues. If you have questions please holler 😄

[chrislovecnm]

@getstek this may be fixed on the master, are you able to compile the latest code and test?? Reference: #6377

[getstek]

I will give it a shot and let you know.

[lesaux]

I compiled master, but I am still not seeing network_interface.0.ipv4_address in the state file on the first run. I did not update go deps, and maybe that was required?

[benlangfeld]

@lesaux I just ran make dev and the resulting binary worked. Maybe you could post a debug log?

[lesaux]

doh! looks like I had a local change in my repo and I didn't notice it when pulled latest. Trying again now.

[lesaux]

indeed this is now fixed. Awesome work @thetuxkeeper !

[ringods]

@stack72 @jen20 Both PR #6377 and #6479 are merged, and people reported it is fixed. I think this should be marked as closed.

[stack72]

Thanks for the reminder @ringods

Closed via #6377 / #6479

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.