-
Notifications
You must be signed in to change notification settings - Fork 455
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Deploying ova template results in endless boot loop #1406
Comments
I think I'm running into the same odd behaviour. network_interface {
network_id = data.vsphere_network.toolsnet.id
} (within the |
Hello @djpbessems . Thank you for your reply. I will give your suggestion a shot and let you know what happens. UPDATE I tried implementing your suggestion. Unfortunately, There was no noticeable change in behavior. |
So... Based on the linked issue, I take away that this is an open issue with no resolution as of yet. Is that correct, @djpbessems ? |
Hi @demdante (and @djpbessems) - Could you please clarify that this issue is with deploying Infoblox NIOS 8.4 with the 825 deployment option and not the nested ESXi 7.0 OVA as the resource name and supporting images suggest? If so, I can attest that the issue still exists when deploying the Infoblox appliance using Terraform. Regards, |
I cannot give you any relevant feedback; I have used neither .ova's you mentioned, sorry. |
Thanks for confirming @djpbessems. Ryan |
Add example for hashicorp/terraform-provider-vsphere#1406.
Hi @demdante -
In Terraform a Example: Variable for the Terraform provider for VMware vSphere configuration. variable "vsphere_insecure" {
type = bool
description = "Allow insecure connections. Set to `true` for self-signed certificates."
default = false
} However, for OVF properties, even though the type is booleen, the vApp Options only accept the values of "True" or "False". In these instances, it's recommended to define the variable as a string in the Terraform plan and pass the value in the correct title case. Example: Variable for Terraform provider for VMware vSphere to pass as an OVF property to an OVA. variable "infoblox_remote_console_enabled" {
type = string
description = "Enable SSH on the virtual appliance. One of `True` or `False`."
default = "True"
} NOTE: You won't be able to get crafty declare it as bool in Terraform and then attempt to convert it to a title case string using title(format("%s", var.infoblox_remote_console_enabled)) as Go is strongly typed. I plan to include these details in a documentation update that I'm submitting per my comment on #1414.
The reason that your NIOS appliance is not booting is related to the SCSI Controller. The default controller in the Terraform provider is <Item>
<rasd:Address>0</rasd:Address>
<rasd:Description>SCSI Controller</rasd:Description>
<rasd:ElementName>scsiController0</rasd:ElementName>
<rasd:InstanceID>3</rasd:InstanceID>
<rasd:ResourceSubType>lsilogic</rasd:ResourceSubType>
<rasd:ResourceType>6</rasd:ResourceType>
</Item> This means that To resolve this issue, add the following: resource "vsphere_virtual_machine" "infoblox" {
# ... other configurations ...
scsi_type = data.vsphere_ovf_vm_template.ovfLocal.scsi_type
# ... other configurations ...
}
There is however one additional complication with this appliance I've not run into with other appliances, whhc is perculiar, but there is a workaround. Even though you have the data source like below....
```hcl
data "vsphere_ovf_vm_template" "ovfLocal" {
name = "foo"
disk_provisioning = var.infoblox_disk_provisioning
resource_pool_id = data.vsphere_resource_pool.default.id
datastore_id = data.vsphere_datastore.datastore.id
host_system_id = data.vsphere_host.host.id
local_ovf_path = var.infoblox_ovf_local
ovf_network_map = {
"VM Network" : data.vsphere_network.network.id
}
} ... and the resource calling the data source like below... resource "vsphere_virtual_machine" "vmFromRemoteOvf" {
# ...other configuration ...
dynamic "network_interface" {
for_each = data.vsphere_ovf_vm_template.ovf.ovf_network_map
content {
network_id = network_interface.value
}
} .. during the deployment you'll see that there are 4 NICs discovered and attached to the To resolve this issue, use the following configuration ... resource "vsphere_virtual_machine" "vmFromRemoteOvf" {
# ...other configuration ...
network_interface {
network_id = data.vsphere_network.network.id
adapter_type = "vmxnet3"
}
network_interface {
network_id = data.vsphere_network.network.id
adapter_type = "vmxnet3"
}
network_interface {
network_id = data.vsphere_network.network.id
adapter_type = "vmxnet3"
}
network_interface {
network_id = data.vsphere_network.network.id
adapter_type = "vmxnet3"
} Results: > terraform apply --auto-approve
Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the
following symbols:
+ create
Terraform will perform the following actions:
# vsphere_virtual_machine.infoblox will be created
+ resource "vsphere_virtual_machine" "infoblox" {
+ boot_retry_delay = 10000
+ change_version = (known after apply)
+ cpu_limit = -1
+ cpu_share_count = (known after apply)
+ cpu_share_level = "normal"
+ datacenter_id = "datacenter-3"
+ datastore_id = "datastore-11"
+ default_ip_address = (known after apply)
+ ept_rvi_mode = "automatic"
+ firmware = "bios"
+ folder = ""
+ force_power_off = true
+ guest_id = "otherGuest64"
+ guest_ip_addresses = (known after apply)
+ hardware_version = (known after apply)
+ host_system_id = "host-10"
+ hv_mode = "hvAuto"
+ id = (known after apply)
+ ide_controller_count = 2
+ imported = (known after apply)
+ latency_sensitivity = "normal"
+ memory = 16384
+ memory_limit = -1
+ memory_share_count = (known after apply)
+ memory_share_level = "normal"
+ migrate_wait_timeout = 30
+ moid = (known after apply)
+ name = "m01-blox01"
+ num_cores_per_socket = 1
+ num_cpus = 2
+ poweron_timeout = 300
+ reboot_required = (known after apply)
+ resource_pool_id = "resgroup-6046"
+ run_tools_scripts_after_power_on = true
+ run_tools_scripts_after_resume = true
+ run_tools_scripts_before_guest_shutdown = true
+ run_tools_scripts_before_guest_standby = true
+ sata_controller_count = 0
+ scsi_bus_sharing = "noSharing"
+ scsi_controller_count = 1
+ scsi_type = "lsilogic"
+ shutdown_wait_timeout = 3
+ storage_policy_id = (known after apply)
+ swap_placement_policy = "inherit"
+ uuid = (known after apply)
+ vapp_transport = (known after apply)
+ vmware_tools_status = (known after apply)
+ vmx_path = (known after apply)
+ wait_for_guest_ip_timeout = 0
+ wait_for_guest_net_routable = true
+ wait_for_guest_net_timeout = 0
+ disk {
+ attach = (known after apply)
+ controller_type = (known after apply)
+ datastore_id = (known after apply)
+ device_address = (known after apply)
+ disk_mode = (known after apply)
+ disk_sharing = (known after apply)
+ eagerly_scrub = (known after apply)
+ io_limit = (known after apply)
+ io_reservation = (known after apply)
+ io_share_count = (known after apply)
+ io_share_level = (known after apply)
+ keep_on_remove = (known after apply)
+ key = (known after apply)
+ label = (known after apply)
+ path = (known after apply)
+ size = (known after apply)
+ storage_policy_id = (known after apply)
+ thin_provisioned = (known after apply)
+ unit_number = (known after apply)
+ uuid = (known after apply)
+ write_through = (known after apply)
}
+ network_interface {
+ adapter_type = "vmxnet3"
+ bandwidth_limit = -1
+ bandwidth_reservation = 0
+ bandwidth_share_count = (known after apply)
+ bandwidth_share_level = "normal"
+ device_address = (known after apply)
+ key = (known after apply)
+ mac_address = (known after apply)
+ network_id = "network-1001"
}
+ network_interface {
+ adapter_type = "vmxnet3"
+ bandwidth_limit = -1
+ bandwidth_reservation = 0
+ bandwidth_share_count = (known after apply)
+ bandwidth_share_level = "normal"
+ device_address = (known after apply)
+ key = (known after apply)
+ mac_address = (known after apply)
+ network_id = "network-1001"
}
+ network_interface {
+ adapter_type = "vmxnet3"
+ bandwidth_limit = -1
+ bandwidth_reservation = 0
+ bandwidth_share_count = (known after apply)
+ bandwidth_share_level = "normal"
+ device_address = (known after apply)
+ key = (known after apply)
+ mac_address = (known after apply)
+ network_id = "network-1001"
}
+ network_interface {
+ adapter_type = "vmxnet3"
+ bandwidth_limit = -1
+ bandwidth_reservation = 0
+ bandwidth_share_count = (known after apply)
+ bandwidth_share_level = "normal"
+ device_address = (known after apply)
+ key = (known after apply)
+ mac_address = (known after apply)
+ network_id = "network-1001"
}
+ ovf_deploy {
+ allow_unverified_ssl_cert = false
+ disk_provisioning = "thin"
+ enable_hidden_properties = false
+ local_ovf_path = "/Users/johnsonryan/Downloads/nios-8.5.2-409296-2021-01-08-00-32-30-ddi.ova"
+ ovf_network_map = {
+ "VM Network" = "network-1001"
}
}
+ vapp {
+ properties = {
+ "default_admin_password" = (sensitive)
+ "lan1-v4_addr" = "172.16.11.132"
+ "lan1-v4_gw" = "172.16.11.1"
+ "lan1-v4_netmask" = "255.255.255.0"
+ "remote_console_enabled" = "True"
+ "temp_license" = "nios IB-V825 enterprise dns dhcp cloud"
}
}
}
Plan: 1 to add, 0 to change, 0 to destroy.
vsphere_virtual_machine.infoblox: Creating...
vsphere_virtual_machine.infoblox: Still creating... [10s elapsed]
# ... time lapse ...
vsphere_virtual_machine.infoblox: Creation complete after 16m53s [id=4202780d-dc91-bd2b-2eba-79a5f64638e3]
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
Outputs:
infoblox_ip_address = "172.16.11.132"
infoblox_mem = 16384
infoblox_name = "m01-blox01"
infoblox_num_cores_per_socket = 1
infoblox_num_cpus = 2
vsphere_datacenter = "m01-dc01"
vsphere_datastore = "local-ssd-01"
vsphere_folder = ""
vsphere_network = "M - 172.16.11.0"
vsphere_resource_pool = "m01-cl01/Resources"
Once the Terraform configuration is correctly set, you'll be able to destroy w/o issue. > terraform destroy --auto-approve
vsphere_virtual_machine.infoblox: Refreshing state... [id=4202780d-dc91-bd2b-2eba-79a5f64638e3]
Note: Objects have changed outside of Terraform
Terraform detected the following changes made outside of Terraform since the last "terraform apply":
# vsphere_virtual_machine.infoblox has been changed
~ resource "vsphere_virtual_machine" "infoblox" {
+ custom_attributes = {}
+ default_ip_address = "169.254.251.1"
+ extra_config = {}
~ guest_ip_addresses = [
+ "169.254.251.1",
+ "172.16.11.132",
+ "fe80::250:56ff:fe82:a507",
]
id = "4202780d-dc91-bd2b-2eba-79a5f64638e3"
name = "m01-blox01"
+ pci_device_id = []
+ tags = []
~ vmware_tools_status = "guestToolsNotRunning" -> "guestToolsRunning"
# (64 unchanged attributes hidden)
# (7 unchanged blocks hidden)
}
Unless you have made equivalent changes to your configuration, or ignored the relevant attributes using ignore_changes,
the following plan may include actions to undo or respond to these changes.
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the
following symbols:
- destroy
Terraform will perform the following actions:
# vsphere_virtual_machine.infoblox will be destroyed
- resource "vsphere_virtual_machine" "infoblox" {
- alternate_guest_name = "" -> null
- annotation = "NIOS automates the error-prone and time-consuming manual tasks associated with deploying and managing DNS, DHCP and IP address management required for continuous IP network availability and business uptime." -> null
- boot_delay = 0 -> null
- boot_retry_delay = 10000 -> null
- boot_retry_enabled = false -> null
- change_version = "2021-11-08T19:21:13.200882Z" -> null
- cpu_hot_add_enabled = false -> null
- cpu_hot_remove_enabled = false -> null
- cpu_limit = -1 -> null
- cpu_performance_counters_enabled = false -> null
- cpu_reservation = 0 -> null
- cpu_share_count = 2000 -> null
- cpu_share_level = "normal" -> null
- custom_attributes = {} -> null
- datacenter_id = "datacenter-3" -> null
- datastore_id = "datastore-11" -> null
- default_ip_address = "169.254.251.1" -> null
- efi_secure_boot_enabled = false -> null
- enable_disk_uuid = false -> null
- enable_logging = false -> null
- ept_rvi_mode = "automatic" -> null
- extra_config = {} -> null
- firmware = "bios" -> null
- folder = "" -> null
- force_power_off = true -> null
- guest_id = "otherGuest64" -> null
- guest_ip_addresses = [
- "169.254.251.1",
- "172.16.11.132",
- "fe80::250:56ff:fe82:a507",
] -> null
- hardware_version = 10 -> null
- host_system_id = "host-10" -> null
- hv_mode = "hvAuto" -> null
- id = "4202780d-dc91-bd2b-2eba-79a5f64638e3" -> null
- ide_controller_count = 2 -> null
- latency_sensitivity = "normal" -> null
- memory = 16384 -> null
- memory_hot_add_enabled = false -> null
- memory_limit = -1 -> null
- memory_reservation = 0 -> null
- memory_share_count = 163840 -> null
- memory_share_level = "normal" -> null
- migrate_wait_timeout = 30 -> null
- moid = "vm-19079" -> null
- name = "m01-blox01" -> null
- nested_hv_enabled = false -> null
- num_cores_per_socket = 1 -> null
- num_cpus = 2 -> null
- pci_device_id = [] -> null
- poweron_timeout = 300 -> null
- reboot_required = false -> null
- resource_pool_id = "resgroup-6046" -> null
- run_tools_scripts_after_power_on = true -> null
- run_tools_scripts_after_resume = true -> null
- run_tools_scripts_before_guest_reboot = false -> null
- run_tools_scripts_before_guest_shutdown = true -> null
- run_tools_scripts_before_guest_standby = true -> null
- sata_controller_count = 0 -> null
- scsi_bus_sharing = "noSharing" -> null
- scsi_controller_count = 1 -> null
- scsi_type = "lsilogic" -> null
- shutdown_wait_timeout = 3 -> null
- storage_policy_id = "" -> null
- swap_placement_policy = "inherit" -> null
- sync_time_with_host = false -> null
- sync_time_with_host_periodically = false -> null
- tags = [] -> null
- uuid = "4202780d-dc91-bd2b-2eba-79a5f64638e3" -> null
- vapp_transport = [
- "iso",
- "com.vmware.guestInfo",
] -> null
- vbs_enabled = false -> null
- vmware_tools_status = "guestToolsRunning" -> null
- vmx_path = "m01-blox01/m01-blox01.vmx" -> null
- vvtd_enabled = false -> null
- wait_for_guest_ip_timeout = 0 -> null
- wait_for_guest_net_routable = true -> null
- wait_for_guest_net_timeout = 0 -> null
- disk {
- attach = false -> null
- controller_type = "scsi" -> null
- datastore_id = "datastore-11" -> null
- device_address = "scsi:0:0" -> null
- disk_mode = "persistent" -> null
- disk_sharing = "sharingNone" -> null
- eagerly_scrub = false -> null
- io_limit = -1 -> null
- io_reservation = 0 -> null
- io_share_count = 1000 -> null
- io_share_level = "normal" -> null
- keep_on_remove = false -> null
- key = 2000 -> null
- label = "disk0" -> null
- path = "m01-blox01/m01-blox01.vmdk" -> null
- size = 250 -> null
- thin_provisioned = true -> null
- unit_number = 0 -> null
- uuid = "6000C292-9c76-1afd-4998-a29bf1f2eb2a" -> null
- write_through = false -> null
}
- network_interface {
- adapter_type = "vmxnet3" -> null
- bandwidth_limit = -1 -> null
- bandwidth_reservation = 0 -> null
- bandwidth_share_count = 50 -> null
- bandwidth_share_level = "normal" -> null
- device_address = "pci:0:7" -> null
- key = 4000 -> null
- mac_address = "00:50:56:82:f3:3b" -> null
- network_id = "network-1001" -> null
- use_static_mac = false -> null
}
- network_interface {
- adapter_type = "vmxnet3" -> null
- bandwidth_limit = -1 -> null
- bandwidth_reservation = 0 -> null
- bandwidth_share_count = 50 -> null
- bandwidth_share_level = "normal" -> null
- device_address = "pci:0:8" -> null
- key = 4001 -> null
- mac_address = "00:50:56:82:a5:07" -> null
- network_id = "network-1001" -> null
- use_static_mac = false -> null
}
- network_interface {
- adapter_type = "vmxnet3" -> null
- bandwidth_limit = -1 -> null
- bandwidth_reservation = 0 -> null
- bandwidth_share_count = 50 -> null
- bandwidth_share_level = "normal" -> null
- device_address = "pci:0:9" -> null
- key = 4002 -> null
- mac_address = "00:50:56:82:5f:3e" -> null
- network_id = "network-1001" -> null
- use_static_mac = false -> null
}
- network_interface {
- adapter_type = "vmxnet3" -> null
- bandwidth_limit = -1 -> null
- bandwidth_reservation = 0 -> null
- bandwidth_share_count = 50 -> null
- bandwidth_share_level = "normal" -> null
- device_address = "pci:0:10" -> null
- key = 4003 -> null
- mac_address = "00:50:56:82:6b:64" -> null
- network_id = "network-1001" -> null
- use_static_mac = false -> null
}
- ovf_deploy {
- allow_unverified_ssl_cert = false -> null
- disk_provisioning = "thin" -> null
- enable_hidden_properties = false -> null
- local_ovf_path = "/Users/johnsonryan/Downloads/nios-8.5.2-409296-2021-01-08-00-32-30-ddi.ova" -> null
- ovf_network_map = {
- "VM Network" = "network-1001"
} -> null
}
- vapp {
- properties = {
- "default_admin_password" = (sensitive)
- "lan1-v4_addr" = "172.16.11.132"
- "lan1-v4_gw" = "172.16.11.1"
- "lan1-v4_netmask" = "255.255.255.0"
- "remote_console_enabled" = "True"
- "temp_license" = "nios IB-V825 enterprise dns dhcp cloud"
} -> null
}
}
Plan: 0 to add, 0 to change, 1 to destroy.
Changes to Outputs:
- infoblox_ip_address = "172.16.11.132" -> null
- infoblox_mem = 16384 -> null
- infoblox_name = "m01-blox01" -> null
- infoblox_num_cores_per_socket = 1 -> null
- infoblox_num_cpus = 2 -> null
- vsphere_datacenter = "m01-dc01" -> null
- vsphere_datastore = "local-ssd-01" -> null
- vsphere_network = "M - 172.16.11.0" -> null
- vsphere_resource_pool = "m01-cl01/Resources" -> null
vsphere_virtual_machine.infoblox: Destroying... [id=4202780d-dc91-bd2b-2eba-79a5f64638e3]
vsphere_virtual_machine.infoblox: Still destroying... [id=4202780d-dc91-bd2b-2eba-79a5f64638e3, 10s elapsed]
vsphere_virtual_machine.infoblox: Still destroying... [id=4202780d-dc91-bd2b-2eba-79a5f64638e3, 20s elapsed]
vsphere_virtual_machine.infoblox: Destruction complete after 28s
Destroy complete! Resources: 1 destroyed.
You can find the full example at https://github.com/tenthirtyam/terrafom-examples-vmware/tree/main/vsphere/vsphere-virtual-machine/ovf-infoblox-ddi. Hope this helps, Recommend: |
@tenthirtyam ah. interesting. thank you for taking the time to look into this. i will give your suggestion a shot, but it looks like you have already confirmed that it is working. thank you very much. |
Happy to help, John. If possible, please mark the issue as closed once you confirm. Take care, |
@demdante - have you had a chance to test yourself? |
Hi @demdante Have you had a chance to test/verify so this issue be closed? Thanks, |
Gentle nudge @demdante. 😁 Ryan |
cc @iBrandyJackson for review and closure consideration. |
Updates `virtual_machine.html.markdown`: - Updated for content accuracy. - Updated all examples to be more readable and complete. - Included continuity between examples. - Addressed concerns mentioned in #1425, #1414, and #1356. - Added some clarify based on issues #1435 and #1406 re: `scsi_type` Signed-off-by: Ryan Johnson <johnsonryan@vmware.com>
Hi @iBrandyJackson and @appilon, this issue appears to be ready for closure based on solution provided. Ryan |
I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further. |
Hello. I have been trying to use terraform and the terraform-provider-vsphere plugin to deploy an ova template in one of our vsphere environments, but am having some difficulty, as the virtual machine which terraform deploys by deploying this particular ova template gets stuck in an endless boot loop, which results in terraform failing because the deployed virtual machine could not register an IP address. However, when I deploy this same ova manually or by using ansible or powercli, this endless boot loop behavior does not occur, leading me to think that there is something about the way this plugin deploys ova/ovf templates that is causing this problem. For your convenience, you can download the ova producing this issue here
Terraform Version
Terraform v0.15.3
vSphere Provider Version
hashicorp/vsphere v1.26.0
Affected Resource(s)
vsphere_virtual_machine
Terraform Configuration Files
Debug Output
https://gist.githubusercontent.com/demdante/e02e9a91abffbd59471e083a4dcdc699/raw/a7be69fdf04f0653ce0d14137120f0a4d1781121/terraform.log
Expected Behavior
Terraform should be able to create a functional virtual machine by deploying the ova template.
Actual Behavior
The virtual machine that is built from the deployed ova gets caught in an endless boot loop, which results in terraform throwing an error because it is not able to detect an ip address from the deployed virtual machine
Steps to Reproduce
terraform apply
Important Factoids
As I already discussed in the previous bullet, including this key value pair will cause terraform to fail with the below error:
But I already explained all of this, so why repeat myself? Well, I mention it because of what happens when you run terraform destroy. When you run terraform destroy after receiving the "timeout waiting for an available IP address" error, terraform is able to destroy the deployed vm successfully. However, if you run terraform destroy after receiving the above "invalid value" error, terraform destroy fails with the below error message:
Terraform will continue to return this error and when attempting to destroy the deployed vm, essentially requiring one to manually go into vsphere and delete the vm from disk, at which point the provider will see the vm is destroyed and will stop complaining. I also captured a debug log for this issue as well, and will include a link to the gist below.
Nic error debug output
https://gist.githubusercontent.com/demdante/46c30e827f15b16f1f44dd62bf029ea2/raw/4e29ee36fb5b79e59abb6d5e5465eb497a0e9063/nicdestroyerror.log
Community Note
The text was updated successfully, but these errors were encountered: