vRealize Orchestrator: Standardising Modules & Actions

Managing your code base in vRealize Orchestrator can be quite challenging and complex. Often, you won’t realise this until you’ve reached a point where it becomes difficult and time consuming to both organise or locate existing code that you have written. In this post, I am going to suggest ways to help you organise your code better, using methods that I have adopted with my time using vRO.

I am not suggesting this be the perfect solution, but it should provide a working standard to adopt to your own needs. I would also argue that the extra time spent getting this in place on the outset, will lead to time saved later on.

Just for reference, from this point on, I am going to refer to ‘code base‘ and ‘actions‘ interchangeably, because your vRO actions ARE your code base. Almost every single line of code you write in vRO should be in an action (I will discuss this in more detail in a later post which I will link here).

Modules

Modules are quite a simple topic but it’s important to get them right as they are almost impossible to change later. Modules are used to organise or group a collection of actions together by a common function.

Here are some general principles I like to follow when creating modules: As a general rule, modules should:

  • Conform to a standard naming convention;
  • Allow developers to easily find existing actions or where to create new actions. Failure to address this point will result in developers re-inventing the wheel by writing new actions for which may already exist;
  • Always be lower case alphabetical characters using dot notation;
  • Always have a utility module for storing general ‘helper’ actions i.e. an action that returns a unique array of items

So generally, if you follow a standard naming convention for your module names then you’ll be set. I have found that the following naming convention works well:

com.[company].library.[component].[interface].[objects].[object]

Where

[company] = company name as a single word

[component] = Examples are: NSX, vCAC, Infoblox, Activedirectory

[interface] – Examples are: REST, SOAP, PS (this is not needed if using a plugin)

[objects] = Parent object branch of the component, i.e. Edges, Entities, Networks, Groups

[object] = A property or object branch of the parent objects. These can contain actions that target a single object (singular)

Here is an example of what (some) of my Zerto Rest API code library looks like:

You could just create a module where the branch stops at [interface] or [objects], but what you’ll find is that it will become a dumping ground for dozens of actions, which will become difficult to maintain. Adding additional branches helps break the modules up.

The strategy around creating these branches usually goes in line with the interface/API’s, which helps align your modules quite nicely. A good module branching strategy can also provide execution performance enhancements by calling the module once for a smaller number of required actions.

Actions

Here are some general principles I like to follow when creating actions: As a general rule, actions should:

  • Contain small, manageable chunks of code that perform a specific task. Actions are just functions and just like any function, it should contain code that performs a specific task. If your action is doing many different tasks, then consider breaking these down into multiple, smaller actions.
  • Validate inputs. I appreciate that some may debate this idea, but actions are not ‘private functions’. They are public code where you can never guarantee that the action ‘caller’ is properly validating its inputs. This is the nature of vRO, it is a ‘hub’ that has many different uses cases and scenarios for executing the same actions. I have seen dozens of cases where developers and support engineers have wasted time tracking unexpected errors;
  • Be named appropriate to the task they perform. I generally like to use verbs in my action names, like, ‘getVirtualMachineNames’, ‘getVirtualMachineNetworks’ or ‘setCustomProperty’. Actions named this way will make it easier for other developers to identify what they are used for;
  • Have variables declared in a single block. This will just make it easier to see what variables are being used. The data type can also be defined, but is not always necessary or as important;
  • Provide consistent logging throughout. Make it so, the action almost tells the story of what is happening. Don’t go overboard, but generally a before, during and after style to logging works quite well;
  • Nesting actions within an action is generally ‘OK’ but keep it to a minimum if possible. Too many nested actions can create depth that may be more difficult to maintain and troubleshoot later on. Typical use cases are ‘helper’ or ‘utility’ actions (you’ll be completely forgiven with actions used for workflow presentation as these are a pain);
  • Perform singular tasks. Don’t write actions that perform plural tasks. Write the singular version first, then use a looping mechanism that re-uses the singular action (there are also ways this can be achieved with performance in mind in vRO). This way you’ll only have 1 version of the code;
  • Be based on a user-defined template. Yup, I’m not crazy. Have a defined template (aka boilerplate) set out on how an action should generally look and have the team follow this. It will make code reviews far easier;
  • Always be camel cased alphabetical characters (no dots);

If you adopt the above principles, you’ll have actions that will be much easier to understand, maintain and troubleshoot and everyone will be a happy bunny.

Action Example

Here is a working example of an action that has been based on a template.

Action Template

And here is the template:

If you’re wondering about the ‘logger‘ action, you can read my post here.

I hope this helps provides some standards around your vRO code base. If you have suggestions, comments, etc. then please let me know as I’ll be glad to hear them and value every bit of feedback.

Install phpVirtualBox on CentOS 7.x

I have to say, I have been bursting with some excitement to recently discover this gem. phpVirtualBox is going to make a great addition to my server by allowing me to create and manage VirtualBox VM’s direct from my web browser. So just a small guide, sticking with the theme, on getting phpVirtualBox installed and running on CentOS 7.x.

Install required packages

Install Apache Web Server

Start Apache and ensure it starts automatically on boot.

Finally, add an exception in the firewall so that you can access Apache over HTTP and HTTPS.

Test that Apache is running and accessible by opening a browser and navigating to http://server_ip_address.

Install PHP

Restart Apache.

Test that PHP is working by creating a PHP test file that will display the PHP information page (watch out for the quotes if copy/pasting).

Verify that the PHP information page loads by opening a browser and navigating to http://server_ip_address/test.php.

Create phpVirtualBox User Account

Create the account that the virtualbox web service will run as and to be used by phpVirtualBox.

Install phpVirtualBox

Download the latest stable phpVirtualBox package (5.2-0-rc1 at the time of writing)

Unzip/Extract phpVirtualBox package.

Configure phpVirtualBox

Edit phpVirtualBox config.php file:

Change the username/password to the user that runs the VirtualBox web service.

Set the user that the VirtualBox web service will run as.

Restart the VirtualBox web service

Add SELinux Policy Exception

If you prefer not to disable SELinux then a policy exception will need to be added that allows phpVirtualBox to connect to the VirtualBox web service. First install the ‘policycoreutils-python’ package, which provides ‘semanage’.

Run the following command to add the exception.

Open a browser and to http://server_ip_address/phpvirtualbox. I’d recommend to use Firefox if you plan on using the console as this uses Flash, which will be blocked by Chrome. Use the default login username: admin and password: admin.

Accessing the Virtual Machine Console Remotely

You will have the capability to access the virtual machine console (to perform assisted installation, access the VM for troubleshooting, etc) using the rdesktop session through phpvirtualbox. You must enable ‘Remote Display’ support on the virtual machine settings, under ‘Display’. The port range defaults to 9000-9100, which should be fine unless you plan to run more VMs :).

In addition to enabling this feature, you will need to add an exception in the firewall to allow your client (web browser) to access the console.

You should now be able to access the VM’s console.

One final note. When connecting to the console, ensure to change the ‘Requested desktop size’ to ‘1024×768’ so that you are able to view the console properly.

Importing Existing VMs/OVA’s Into VirtualBox

Before I decided to run a Linux based, headless installation of VirtualBox, I had been running all my virtual machines in VMware Workstation. When it was time to switch I exported a number of virtual machines that I had already built to OVF format. These were servers like Windows with Active Directory, GIT, Ansible, that I didn’t want to go through the hassle of building again from scratch. This post is dedicated to my experience of doing this and some of the post migration work that needs to be performed, such as removing VMware Tools, etc.

Import Existing Virtual Machine Image

Before you get started

One tip before you start. When you import the VM into VirtualBox, it’s possible that the network adaptor information will change. This was definitely the case for me when I migrated from Workstation that used the VMXNET3 driver. My interface was configured as ‘ens33‘ and when I switched to the virtio driver in VirtualBox the device came up as eth0. I was therefore unable to connect to my headless VM. My solution was to fire up the source VM again and copy the ‘ifcfg-ens33‘ in /etc/sysconfig/network-scripts/ to ‘ifcfg-eth0‘ and change any references from ens33 to eth0. Finally, I commented out the UUID line. When I re-imported, I was able to connect the my VM.

Import Virtual Machine

To perform the import I am going to use the VBoxManage import command. This command allows the –dry-run flag to be supplied and will provide details of how the virtual machine will look once it has been imported into VirtualBox. This is useful as it allows us to ensure that the configuration is how we want it to be. The dry run will also provide any optional flags that can be used to influence the import. There are also the global options keepallmacs, keepnatmacs, and importtovdi.

I have some virtual machines that have been exported to OVF format:

So let’s take a look at my Ansible (SG1-ANS001) virtual machine that is running on CentOS 7.x.

The dry run feature represents the virtual machine image as a Virtual system followed by an index. As this image only contains a single virtual machine, only index 0 is present. The VM hardware configuration is listed as units. Therefore, I can go through each unit in turn and ensure that the target machine is configured correctly. From the dry run output there are a number of units that need to be modified. I will build out the command as I go along.

1: Suggested VM name “vm” (change with “–vsys 0 –vmname <name>”)

This will need to be changed to the actual name of the VM, which in this case is SG1-ANS001. I can use the options –vsys 0 –vmname <name> to accomplish this.

5: Network adapter: orig custom, config 5, extra type=Bridged

The original VM was using a custom host-only network interface on the previous hypervisor which is not understood by VirtualBox. The import command is therefore defaulting to a suggestion of a Bridged interface, which is not correct. I need connect the virtual network card to my vboxnet0 host-only interface on VirtualBox. Unfortunately, the import command does not provide an option that I can see which can make this change. I will therefore need to modify the VM once it has been imported.

9: Hard disk image: source image=SG1-ANS001-disk1.vmdk, target path=/mnt/vm2/vbox/vm/vm/SG1-ANS001-disk1.vmdk, controller=7;channel=0

The source disk image is a VMDK, the format used by VMware. Although VirtualBox uses the VDI disk image format, it has defaulted to continue using VMDK. This will still work but as my goal is to move to a fully Open Source solution, converting to VDI makes more sense. The global option importtovdi can be used to achieve this. My command now looks like:

VirtualBox will default to re-initialising the MAC address of all network cards on the VM. I am not interesting in making post configuration changes because of this and my preference would be to preserve the existing MAC addresses that was allocated to the original VM. There is no real reason not to do this. The global option keepallmacs can be used to achieve this. My final command now looks like this:

If we issue another –dry-run against the new command we can see most of the changes have taken effect.

We can now re-issue the command with the –dry-run omitted to perform the import of this virtual machine into VirtualBox. The progress will be displayed and a message if the import was successful:

Post-Import Configuration

Let’s check that we can see the VM in the VirtualBox inventory:

Configure Networking

If you recall from the previous section, we were not able to change the network card binding during the import. We can do this now that the VM has been imported using the VBoxManage modifyvm command. I need to bind to a host-only interface called vboxnet0.

I also want to use the paravirtualized Network Adaptor using the virtio driver for better performance.

The VBoxManage showvminfo command can be used to view the VMs configuration, which can be used to confirm any changes that have been made.

Disable Audio

I noticed from my VBoxManage showvminfo output that an Audio card was enabled.

Audio: enabled (Driver: ALSA, Controller: AC97, Codec: STAC9700)

I do not require this so the card can be disabled:

Set VM Description

While we’re at it, it would probably be a good idea to give this VM a small description.

Virtual Machine Power Operations

Start VM

Since we’re going to be running this VM in headless mode, the alternative binary VBoxHeadless will be used instead of VBoxManage. The VBoxHeadless interface accepts the same start parameters. If you see the copyright information after you have run the command then the VM will have been started successfully.

Use the  VBoxManage list runningvms command to verify that the VM process is actually running.

Stop VM

You can gracefully stop the running VM using the VBoxManage controlvm <vm> acpipowerbutton command.

Troubleshooting

If you aren’t able to connect to the VM after it has been started then it may have failed to boot or a network configuration issue. As the VMs are running in headless mode it can be difficult to diagnose. There are a couple of decent ways to help diagnose the issue.

Screenshot

This option is simple and takes a screenshot of the console. You can specify the filename to save (in PNG format) then download this via SCP from the VirtualBox host.

Serial Port (console redirect)

I will follow up with an additional post on how the display output can be redirected to the serial port and some cool ways that we can view and interact with this session.

If there is some other configuration that I could apply to my VMs or vbox installation, then please let me know.

CentOS running VirtualBox (headless mode)

Today I started work on something that has peeked my interest for a while, switching my server to CentOS running VirtualBox in headless mode.

I had long been a fan of VMware Workstation, back in the day, it was more feature rich than Vbox and provided better memory management features. Alas, that is not the case today and with VirtualBox’s powerful ‘VBoxManage‘ CLI, it really fits in well where I can write all my infrastructure in code (and yes, I’ll most likely layer Vagrant on top of this). VirtualBox also provides an alternative headless interface ‘VBoxHeadless‘, which means there is no requirement to run a GUI on my server.

As I am starting out on this new journey I felt that it would be great to blog about and hopefully help others that want to do this.

All I have from the start is a minimal installation of CentOS 7.4 (Core). I am using the following as a guide for the VirtualBox installation: https://wiki.centos.org/HowTos/Virtualization/VirtualBox.

VirtualBox Installation

Install Dependencies

Install Extra Packages for Enterprise Linux (EPEL)

Install Dynamic Kernel Module Support (DKMS)

This will install quite a few packages:

Install Development Tools

I want my server to have access to a basic development environment so we’ll install the group packages for this. This will install packages such as gcc, make, binutils, etc. Use ‘yum groupinfo “Development Tools“‘ to view the entire list of packages installed in this group.

Install Kernel Development

Install VirtualBox

Add the VirtualBox package repository

Install VirtualBox

This will also install a number of dependencies.

Install VirtualBox Extension Pack

The VirtualBox Extension Pack will add support for the following:

  • USB 2.0 and USB 3.0 Host Controller
  • Host Webcam
  • VirtualBox RDP
  • PXE ROM
  • Disk Encryption
  • NVMe

Download the extension pack:

Once downloaded, install the extension pack:

OR, if you are upgrading from a previous version of the extension pack, then you will need to add the ‘–replace’ option to uninstall the old version first.

Agree to the license terms and conditions.

The extension will be installed.

Verify that the extension pack has been installed successfully.

You should see output like the following:

Add Users to ‘vboxusers’ Group

When VirtualBox is installed a new group ‘vboxusers’ is created. Users that are a member of this group will be allowed to run VirtualBox. I will add my non-privileged user to this group.

Verify Installation

VirtualBox Linux kernel module (vboxdrv)

Check that the vboxdrv service has installed correctly and is running.

You should see output like the following:

The status should be ‘loaded‘ and ‘active‘. If you see a ‘Kernel driver not installed‘ message then try running the following command:

VirtualBox Web Service (vboxweb)

Check that the vboxweb service is running:

You should see output like the following:

Configure VirtualBox

Configure Networking

I have some basic network requirements for my lab to start out with. I will use a host-only interface, which will be the management network and a NAT interface, which can be useful for VMs that I want to access the Internet without going through the virtual firewall. I will also need to bridge one of my virtual firewall’s network interfaces to the servers physical interface (so that I can do some additional routing on my physical network).

I can see what network interfaces are currently available using nmcli.

Create the Host-Only Interface

We can confirm that the Host-Only interface was successfully created using nmcli.

Assign Network to Host-Only Interface

To confirm that the IP address has been assigned, use the ip command.

Create NAT Interface

I want to create a NAT network on 192.168.15.0/24 and have IP addresses allocated to clients automatically using DHCP.

To confirm that the NAT network has been created use the following command.

We can see that the network has been created and an IP address has been allocated automatically for the gateway (uses the first available address). This should be enough to provide outbound Internet access to any VM NICs attached to this interface.

Global Settings

I also like to configure some global settings so that I do not need to keep specifying these when creating new virtual machines.

Default Virtual Machine Folder

I actually have 2 large SSDs that I spread the VMs across for IO reasons. I will default to one of these and manually specificity my second disk when required.

Exclusive Hardware Virtualization

VirtualBox will be given exclusive use of the hardware virtualization extensions (Intel VT-x or AMD-V). I think this defaults to on but let’s set it anyway.

Default Front End

I am not running a GUI on this server so all virtual machines will be running in headless mode.

That concludes my initial setup and now I am ready to start deploying my virtual machines. I will provide posts on my new virtual machine deployments as I build out my new infrastructure using VirtualBox. I also have a number of OVF’s that I exported from my old environment that I’ll be importing and will document the steps along the way.

Using the indexOf() method for Arrays and Strings for vRO and vRA

I’ve never been a developer so getting into JavaScript was quite a challenge at first and I probably always went the longest route possible to achieve something. As I use it more and more, I am picking up these neat little tricks and uses for built in methods that make my life easier.

In the world of vRA and vRO, I find that most of my time is spent iterating over arrays or parsing custom properties. One method that I have come to find extremely useful is the indexOf() that is available on Arrays and Strings. The methods are very similar but have very different use cases. Let’s take a look at each of them in turn.

String indexOf() Method

w3schools.com defines this as:

The indexOf() method returns the position of the first occurrence of a specified value in a string.

This method returns -1 if the value to search for never occurs.

So as an example, if we had the string “simplygeek.co.uk is fun”

string.indexOf(“m”) would return 2, which is the index within the string that ‘m’ first appears. Note that if ‘m’ appeared twice then only the first match would return a result. Indexes within arrays and strings always start at 0.

Another use case, one which I find the most useful, is being able to provide a string for the lookup. Take the following example:

string.indexOf(“simplygeek”) would return 0, because in a contiguous match the first index is returned.

When writing JavaScript that interacts with vRA you are often required to parse through custom properties, which are key value pairs of data. Such properties can contain useful information that relates to a deployment, such as virtual machine configuration. If custom properties follow a standardised naming convention, it can be easy to discover a set of properties. Let’s assume I have created the following custom properties in vRA for a deployment:

Custom.Deployment.Virtualmachine.Config.hotcpu : true
Custom.Deployment.Virtualmachine.Config.hotmem : true
Custom.Deployment.Virtualmachine.Config.sched.swap.vmxSwapEnabled : true

When the payload is sent to my vRO workflow it could contain over a 100 different key:value pairs of data. To find these easily I can use the indexOf method to iterate over each pair as follows:

The above will result in an array of properties related to virtual machine configs. I can then pass this array to some code that will handle the implementation of these advanced virtual machine settings. This allows for a very dynamic way to manage custom properties in property groups within vRA.

Array indexOf() method

Very similar to the String method, on an array, the indexOf() method returns the first index at which a given element can be found in the array, or -1 if it is not present. I find this method useful when I need to return a set of unique values from another array. let’s assume we have the following array:

myArray = [‘one’, ‘two’, ‘two’, ‘three’, ‘three’, ‘three’]

If I wanted to return only unique items from myArray, I could use the indexOf method as follows:

The above code will result in an array:

[‘one’, ‘two’, ‘three’]

I hope that someone else finds these as useful as I have. If you know of more use cases, then please let me know.