vRealize Automation Virtual Machine is Missing after being Moved or Recovered

Have you ever seen a deployed virtual machine in the vRA portal have its status changed to ‘Missing‘? This can happen for two reasons; The first, obvious reason, is that the source virtual machine has been deleted. Not much you can do about that except re-provision it or restore the vm from backup. The second reason, is because the reference information that vRA holds about the virtual machine is no longer valid (perhaps from an earlier restore).

In this post, I will discuss and provide a solution that will address the latter for a vSphere hosted virtual machine.

When in this state, you will not be able to perform day 2 actions on this virtual machine (apart from destroy).

The following scenarios can explain why vRA has incorrect reference data about the virtual machine:

  • The virtual machine has been recovered from backup and a new instance uuid has been created;
  • The virtual machine has moved to another vCenter Server, maybe during a DR event. I know that Zerto will create a new virtual machine at the DR site which will have a different instance uuid (SRM’s shadow VMs are most likely similar but I can’t confirm);
  • The virtual machine was cloned and the original has been deleted;
  • The instance uuid for the virtual machine has been changed, either manually or by some other process.

All code that I have provided or talked about in this post can be downloaded as a vRO package for your consumption here.

How vRA References the vCenter Virtual Machine

The virtual machine entity in vRA, is simply a record in the database, with information about that virtual machine (such as description, vCPUs, memory, etc.) and relationships it has with other objects, such as a reservation. Importantly, the record also contains some fields, which provide references and/or unique identifiers to the virtual machine on the platform it was deployed on. In the case of vSphere, the virtual machine id (moref) and instance uuid is stored.

When a virtual machine has been recovered, or any process which causes these values to differ from what vRA has, will result in the virtual machine being displayed as ‘Missing‘.

There is also an internal custom property called “VirtualMachine.Admin.UUID” that also stores the instance uuid and would need to be updated.

Updating vRA Virtual Machine References

I have created a workflow called ‘Pair vRA VM with vCenter VM‘ that will allow you to select the vRA virtual machine and then select the vCenter virtual machine that it should be paired with. from the vRO inventory.

The solution includes running 2 workflows. The first workflow, ‘Update vRA VM Pointers for VcVm‘, will allow you to select the vRA virtual machine and the vCenter virtual machine that it should be paired with. Once this has completed, the status of the virtual machine in vRA should display the correct status and you will be able to manage it as normal.

When you run the workflow you will be able to select the virtual machines that you want to pair from the vRO inventory.

In some cases you might see a duplicated virtual machine in the vRA inventory. This happens when the data collection process creates a new entity for the virtual machine that it found on vCenter. The reason for this is that the new virtual machine has a different uuid.

When selecting the vRA virtual machine, make sure to select the one that has the ‘isMissing‘ property set to ‘true‘, as this is the one you see in the vRA portal.

As you can see in the screenshot above, I have 2 entities with the same VM name, but only one of these has the ‘isMissing‘ property set to ‘true‘.

When the workflow runs, it is going to process the following high level steps:

  • Updates the VMUniqueID, ExternalReferenceId and sets the OwnerExists property to true.
  • Updates the ‘VirtualMachine.Admin.UUID’ custom property.
  • Checks for duplicate/invalid vRA VM entities (see above) and removes these.
  • Runs a data collection if at least one invalid entity has been found (as these can still cause the VM to be displayed as missing).

Below is the log output from the workflow:

When I refresh the browser I can now see the correct status for the virtual machine and perform day 2 actions as normal.

We can also see that there is now only one virtual machine in the vRO inventory (the invalid entry was removed):

I hope this has been helpful. If you discover bugs with any of my code, require some help or simply need an ad-hoc solution, then please drop me a message via the Drift app.

Please rate this post!

Average rating / 5. Vote count:

Leave a Reply

avatar
  Subscribe  
Notify of