vRO Coding Pet Hate – Badly formed IF statements

I was optimising some vRO actions today and came across something that always annoys me when I see it, badly formed IF statements (for one of a better description). When I write my code, I have learned that formatting and consistency is key to produce code that is easier to read and understand. I also like to keep things simple. However, I am constantly surprised at how much sloppy coding practices I encounter from the code that is shipped with vRO. Here are a couple of small examples of the kind of thing I am talking about.

if (ip == null) throw  "ReferenceError: ip cannot be null";
if (netBIOS != null)
    ipSettings.netBIOS = VcCustomizationNetBIOSMode.fromString(netBIOS.name) ;

Both of the above statements are pretty much identical in how they function. They are checking if a given variable is null (or not) and if the condition returns ‘true’ then execute the following statement. As you can see from the code snippets above, the structure of these statements are different. Why? Because this is a valid way to write conditionals in JavaScript, if you are only performing an action when the result is ‘true’. That said, it doesn’t mean we should use it in practice. The reason for this, is that it forces a lack of consistency in your code (which you can already see from the code above).

If we wanted to add another action for a result that returns ‘false’, we would need to add an ‘else’ clause. With the conditional statements above, we wouldn’t be able to do this as it would require adding curly braces so that the interpreter knows when to apply the correct code based on the result. Here is an example.

if (condition) {
    execute_if_true;
} else {
    execute_if_false;
}

This is the structure that I like to adhere to every time I write a conditional. I also apply the ‘4 spaces rule’ for my indentation (and don’t get me started on the use of tab!). The two code snippets revised using this structure would look like:

if (ip == null) {
    throw  "ReferenceError: IP cannot be null";
}
if (netBIOS != null) {
    ipSettings.netBIOS = VcCustomizationNetBIOSMode.fromString(netBIOS.name);
}

I believe this is a much neater approach for the sake of a few more lines, which aren’t worth saving if the code is more readable.

Get a list of Datastores, quickly, using XPATH

The vCenter plugin has a very useful method for retrieving a list of objects quickly and easily. However, if your inventory is quite large, then this can be quite slow. There are two ways that this can be improved. 1) only retrieve the attributes that are useful for our code requirements, 2) use an XPATH query filter to limit the scope of the search.

In my situation there were over a 100 Datastores but I only wanted to retrieve a list based on a given prefix (of which there were less than 10). The Datastore names end with an incrementing number and I wanted to return all of these in an array.

// Set the Datastore name Prefix to search on.
var DsNamePrefix = "DS_MYDATASTORE_";

// Declare which Datastore properties we care about to speed up processing.
var DsProps = ['info', 'summary'];
// Setting the XPATH massively improves performance retrieving the Datastore list.
var XPath = "xpath:@name[starts-with(.,'" + DsNamePrefix + "')]";
	
// Get an array of all the datastores found.
var VcDsArray = VcPlugin.getAllDatastores(DsProps, XPath);
	
System.debug("Number of datastores found: " + VcDsArray.length);

I have used a similar approach which allowed me to retrieve several thousand virtual machines in approximately 3-5 seconds.

Get Datastore with Most Free Space & Check Datastore Meets Capacity Reservation

I do a lot of work that involves either creating new virtual hard disks or attaching existing ones to virtual machines in VMware vSphere. I do all of this through vRealize Orchestrator, written in JavaScript (yum).

As part of this task, I always ensure that the datastores that I am creating disks on have sufficient capacity for these new disks. I use two simple functions that I have written which perform these checks for me, which is ‘getDatastoreWithMostFreeSpace’ and ‘DoesDSMeetCapcityReserve’. An additional function ‘convertToGB’, which is used by these functions is also provided at the end.

Function : getDatastoreWithMostFreeSpace

Description
Return a single VcDatastore object that has the most free space from an Array of provided VcDatastore objects. The function will always return a VcDatastore, regardless of how much free space is found.

Parameters
– VcDatastores (Array of VcDatastore)

Return Type : VcDatastore

function getDatastoreWithMostFreeSpace(VcDatastores) {
    // Initilize variables.
    var DsWithMostFreeSpace = "";
	var maxFreeSpaceFound = 0;
	
	System.log("Selecting the datastore with the most free space:");
	// Loop through each VcDatastore in the Array.
	for (d in VcDatastores) {
	    // Set dsIndex to an increment of the array index (nicer than seeing numbers that start from 0).
	    var dsIndex = Number(d+1);
	    // Check if the Datastore is in maintenance mode.
	    if (VcDatastores[d].summary.maintenanceMode === 'normal') {
	        // Update free space and capacity usage information for the current Datastore.
            VcDatastores[d].refreshDatastore();
            // Output Datastore info.
		    System.log("Datastore " + dsIndex + " information:");
		    System.log("\tname: " + VcDatastores[d].name);
			System.log("\tID: " + VcDatastores[d].id);
			
			// Get Datastore Capacity value.
		    var DsCapacityGB = convertToGB(VcDatastores[d].summary.capacity);
		    System.log("\tCapacity: " + DsCapacityGB + " GB");
		
		    // Get Datastore Free Space value.
			var DsFreeSpaceGB = convertToGB(VcDatastores[d].summary.freeSpace);
		    System.log("\tFree Space: " + DsFreeSpaceGB + " GB");
		
		    // Get Datastore Used Space value.
		    var DsUsedSpaceGB = DsCapacityGB - DsFreeSpaceGB;
		    System.log("\tUsed Space: " + DsUsedSpaceGB + " GB");
		
		    // Get maximum supported VMDK size.
		    var DsMaxVMDKSizeGB = convertToGB(VcDatastores[d].info.maxVirtualDiskCapacity);
		    System.log("\tMaxVMDKSize: " + DsMaxVMDKSizeGB + " GB");
		
		    // Check if the current datastore has the most free space and remember this datastore.
		    if (DsFreeSpaceGB > maxFreeSpaceFound) {
		        maxFreeSpaceFound = DsFreeSpaceGB;
				DsWithMostFreeSpace = VcDatastores[d];
		    } // End if
        } else {
            // Output a warning if current datastore is currently in maintenance mode.
            System.warn("Datastore: '" + DsWithMostFreeSpace.name + "' in state: '" + VcDatastores[d].summary.maintenanceMode + "'. Skipping.");
            break;
        } //End if
	} // End for
	
    // Return a single VcDatastore object.
	return DsWithMostFreeSpace;
}

Function : DoesDSMeetCapcityReserve

Description
Checks that a given VcDatastore has enough free space in which to place the virtual disk, taking into account current usage and a defined reservation value (in percent) that should be guaranteed. Returns true if the Datastore passes the capacity checks.

Parameters
– datastore (VcDatastore)
– reservation (Number)
– totalSizeRequired (Number)

Return Type : Boolean

function DoesDSMeetCapcityReserve(datastore, reservation, totalSizeRequired) {
    // Set meetsReservation to default to false.
    var meetsReservation = false;
    var DsCapacityGB = convertToGB(datastore.summary.capacity);
    var DsFreeSpaceGB = convertToGB(datastore.summary.freeSpace);
    // Get the percentage of free space available.
    var DsPercentFree = Math.ceil((DsFreeSpaceGB/DsCapacityGB)*100);
    System.log("Datastore reservation is set to " + reservation + "%");
    System.log("Datastore has: " + DsPercentFree + "% free space");

    if (DsPercentFree > reservation) {
        System.log("Datastore meets " + reservation + "% reservation threshold.")
        System.log("Checking that enough storage is available for the disks");
        var reservationSizeGB = Math.ceil((DsCapacityGB/100)*reservation);

        System.log("The reserved size at " + reservation +  "% is: " + reservationSizeGB + " GB");
        
        var freeSpaceWithoutReserve = (DsFreeSpaceGB - reservationSizeGB);
        System.log("Free space available taking into account the reservation: " + freeSpaceWithoutReserve + " GB");

        var freeSpaceAfterCommit = (freeSpaceWithoutReserve - totalSizeRequired);
        System.log("The datastore will have " + freeSpaceAfterCommit + " GB of usable space remaining after the commit");

        if (freeSpaceAfterCommit > 0) {
            System.log("Datastore meets capacity requirements");
            meetsReservation = true;
        }
    };
    return meetsReservation;
}

Function : convertToGB

Description
Converts a value in Bytes to GB.

Parameters
– size (Number)

Return Type : Number

function convertToGB(size) {
    sizeGB = Math.ceil(size/1024/1024/1024);
    return sizeGB;
}

Example Output

[2017-09-18 15:09:37.459] [I] Selecting the datastore with the most free space:
[2017-09-18 15:09:37.714] [I] Datastore 1 information:
[2017-09-18 15:09:37.715] [I] name: DATASTORE1
[2017-09-18 15:09:37.717] [I] ID: datastore-99999
[2017-09-18 15:09:37.718] [I] Capacity: 2048 GB
[2017-09-18 15:09:37.719] [I] Free Space: 1762 GB
[2017-09-18 15:09:37.721] [I] Used Space: 286 GB
[2017-09-18 15:09:37.725] [I] MaxVMDKSize: 63488 GB
[2017-09-18 15:09:37.726] [I] Selected Candidate Datastore: DATASTORE1
[2017-09-18 15:09:37.728] [I] Performing capacity validation checks on the datastore.
[2017-09-18 15:09:37.729] [I] Datastore reservation is set to 10%
[2017-09-18 15:09:37.730] [I] Datastore has: 87% free space
[2017-09-18 15:09:37.733] [I] Datastore meets 10% reservation threshold.
[2017-09-18 15:09:37.735] [I] Checking that enough storage is available for the virtual disk(s)
[2017-09-18 15:09:37.737] [I] The reserved size at 10% is: 205 GB
[2017-09-18 15:09:37.738] [I] Free space available taking into account the reservation: 1557 GB
[2017-09-18 15:09:37.740] [I] The datastore will have 1556 GB of usable space remaining after the commit
[2017-09-18 15:09:37.741] [I] Datastore meets capacity requirements
[2017-09-18 15:09:37.744] [I] The Virtual Disk(s) will be located on: DATASTORE1

vRA 7 – Getting more information in workflows from the vRO ExecutionContext object

When a vRO workflow or action is called from vRA, additional input parameters (in addition to those specified as the workflow/action inputs) are provided in the Execution Context object of the workflow. These can be very useful as they contain additional data that can be used inside the workflows. A couple of good examples would be the user that requested the resource or the name of the tenant.

All code that I have provided or talked about in this post can be downloaded as a vRO package for your consumption here.

Parameters Available

Here is a list of parameters provided inside the Execution Context (those without a description I’m still trying to figure out) of a workflow that has been executed by the Event Broker during VM provisioning.

 Parameter Name  Description
__asd_catalogRequestId The Request ID (vCAC Entity object reference)
__asd_correlationId  The Request ID (vCACCAFE object reference)
__asd_requestInstanceId
__asd_requestInstanceTimestamp Request Date & Time (UTC)
__asd_requestInstanceTypeId The lifecycle event topic id i.e. com.vmware.csp.iaas.blueprint.service.machine.lifecycle.provision
__asd_requestTraceId
__asd_requestedBy The UPN of the user who made the request
__asd_requestedFor The UPN of the user the request was on behalf of
__asd_targetResourceId The vCAC virtual machine entity id
__asd_targetResourceTypeId Type ID, i.e. ‘machine’
__asd_targetResourceProviderId
__asd_targetResourceProviderTypeId com.vmware.csp.iaas.blueprint.service
__asd_tenantRef The id of the tenant (same id which goes in the url)

If anyone can shed some light on what the other parameters do then please comment and I’ll update the page.

You can also see these in the variables tab for the workflow run:

There is also another parameter which does not appear:

 Parameter Name  Description
__asd_subtenantRef The business Group id

This parameter is used to identify the business group which the user who submitted the request is a member of. For some reason this isn’t made available like the above parameters but only becomes available when you create and submit an XaaS Blueprint (a catalogue request mapped directly to a workflow).

The only workaround that I have used to get Business Group information available in Event Broker subscriptions is to create a custom property on each business group called ‘Custom.Business.Group.Name’ with the string name of the business group. It’s not a great solution but it does allow the custom property to be queried in the usual way.

Retrieve Parameter Values

There are two ways that you can retrieve the values from these parameters. The first is to simply add these parameters as an input to the workflow. vRA will automatically detect and populate these parameters with the corresponding values. You can then use these inputs as you normally would in your workflow.

The second option is to retrieve them from the Execution Context object.

Workflow Execution Context

The parameters and their values in the execution context object can be retrieved using the System scripting class. There is a method called ‘getContext()‘, which returns an object called ‘ExecutionContext‘. The ExecutionContext object has a ‘getParameter’ method which can be used to retrieve the value for the specified parameter.

Insert the following code into a scripting task in your workflow to list all the parameters and their values:

var executionContext = System.getContext();

System.log("\nExecution context Parameters:"
for each (var parameter in executionContext.parameterNames().sort()) {
    System.log(parameter + " : " + executionContext.getParameter(parameter))
}

Retrieve the Value for a Specific Parameter

To retrieve a value for a specified parameter the following example can be used:

Retrieves the tenant identifier and sets it to a variable ‘tenant’.

var tenant = System.getContext().getParameter("__asd_tenantRef");

Retrieve Parameter Values using Actions

I always like to do everything using Actions as I like to write my code once and simply re-use them repeatedly by dropping them onto my workflows. I have created a collection of Actions that can be used to get values from the Execution Context. As always, a link is provided to all my code at the beginning of this page.

I have a ‘core’ helper Action that will do the main work of retrieving the parameter value. This is a generic Action for getting a value from the Execution Context and is not directly related to vRA.

getExecutionContextParameterValue

/*global parameter*/

/**
 * Retrieves the value for the specified parameter in the Execution Context.
 * @author Gavin Stephens <gavin.stephens@simplygeek.co.uk>
 * @version 1.2.0
 * @function getExecutionContextParameterValue
 * @param {string} parameter - The Execution Context parameter.
 * @returns {string} Returns the parameter value.
 */

function checkParams(parameter) {
    var inputErrors = [];
    var errorMessage;
    if (!parameter || typeof parameter !== "string") {
        inputErrors.push(" - parameter missing or not of type 'string'");
    }
    if (inputErrors.length > 0) {
        errorMessage = "Mandatory parameters not satisfied:\n" + inputErrors.join("\n");
        log.error(errorMessage);
    }
}

var logType = "Action";
var logName = "getExecutionContextParameterValue"; // This must be set to the name of the action
var Logger = System.getModule("com.simplygeek.library.util").logger(logType, logName);
var log = new Logger(logType, logName);
var paramValue = "";

try {
    checkParams(parameter);
    log.debug("Getting value for parameter '" + parameter + "'");
    paramValue = System.getContext().getParameter(parameter);
    if (paramValue) {
        log.debug("Found parameter value: " + paramValue);
    } else {
        log.error("Could not find value for parameter '" + parameter + "'");
    }
} catch (e) {
    log.error("Action failed to get parameter value.",e);
}

return paramValue;

The following Action is a wrapper for the ‘getExecutionContextParameterValue‘, that is used to get a value for a specific parameter.

getTenantRefParameterValue

/**
 * Returns the value for the parameter '__asd_tenantRef'.
 * @author Gavin Stephens <gavin.stephens@simplygeek.co.uk>
 * @version 1.1.0
 * @function getTenantRefParameterValue
 * @returns {string} Returns the parameter value.
 */

var logType = "Action";
var logName = "getTenantRefParameterValue"; // This must be set to the name of the action
var Logger = System.getModule("com.simplygeek.library.util").logger(logType, logName);
var log = new Logger(logType, logName);
var parameter = "__asd_tenantRef";
var paramValue;

try {
    log.log("Getting value for parameter '" + parameter + "'");
    paramValue = System.getModule("com.simplygeek.library.vro.workflow").getExecutionContextParameterValue(parameter);
    if (!paramValue) {
        log.error("No value was found for parameter '" + parameter + "'");
    }
    log.log("Found parameter value '" + paramValue + "'");
} catch (e) {
    log.error("Action failed to get value for parameter '" + parameter + "'",e);
}

return paramValue;
[2019-01-31 22:47:07.909] [I] [Action: getTenantRefParameterValue] Getting value for parameter '__asd_tenantRef'
[2019-01-31 22:47:07.926] [D] [Action: getExecutionContextParameterValue] Getting value for parameter '__asd_tenantRef'
[2019-01-31 22:47:07.929] [D] [Action: getExecutionContextParameterValue] Found parameter value: sg
[2019-01-31 22:47:07.934] [I] [Action: getTenantRefParameterValue] Found parameter value 'sg'

You would simply drop this Action on to your workflow and set the output to store the parameter value.

I have included all the above Actions plus those required to retrieve all parameter values.

As always, if you need any help then please drop me a message via the Drift app.

Extend vRA UI Login Timeout

Nothing frustrates me more than returning to the vRA UI, clicking something and then have it immediately log me out. This appears to be quite random a lot of the time and whilst I can appreciate that this is in the interest of security, I find it a little too aggressive. This can be changed by extending the life of the authentication cookie using the following procedure.

Log in to the vRA appliance using SSH (SSH will need to be enabled via the vRA Appliance VAMI interface if it is not already). Edit the following file:

vi /etc/vcac/setenv-core

Add the following line to the end of the file:

VCAC_OPTS="$VCAC_OPTS -Dvcac.identity.auth.cookie.ttl=28800"

Where ‘28800’ is the number of seconds (8 hours) until the authentication cookie expires. Set this based on your own requirements.

Save the file and then restart the vcac-server service

service vcac-server restart

It will take the usual 5 minutes or so for the server to fully start up and once complete the new settings will be applied.

Repeat this process for any remaining appliances in the cluster.

vRealize Orchestrator cluster nodes not in sync with embedded vRA Appliance

I have come across this issue a number of times where the secondary vRO cluster node is not in sync with the primary node (one thing to note is that the synchronisation status is relative to the node you are logged on to).

The option to synchronise the nodes is not available because I am using vRO that comes embedded with the vRA appliance. The reason this option is not available is because vRA “should” be managing the state of the cluster nodes, which is does from a vRO client prospective but not when you make changes via the Control Center (such as changing the Admins group).

To workaround this issue and unlock the hidden options you will need to append “?advanced” to the end of the URL. For example, if you are on the Orchestrator Cluster Management page add ?advanced to the end of the URL, which should look like this:

https://vro_server:8283/vco-controlcenter/#/control-app/ha?advanced

When the page refreshes you will notice that a new button has appeared on the bottom.

Clicking on this button will reveal two new options:

  • Push Configuration
  • Push Configuration and restart nodes

Select “Push Configuration and restart nodes”, which will push the configuration and automatically restart the vRO service on the secondary node.

A message will be displayed if this is successful.

It will take approx 10 minutes for the secondary node to start up completely so grab a coffee at this point.

Refresh and both nodes should now show as “Synchronized”.

Remove ‘Default Machine Prefix’ for a Business Group

When I was building vRA 7 and figuring everything out, at some point I set the ‘Default machine prefix‘ for the Business Groups. When you first create a Business Group, setting the default machine prefix is optional; However, if you do set this value then there is no way to unset it, which I find extremely annoying. As I don’t currently use specific prefixes for business groups, I was feeling a little OCD having to set this to some other prefix value.

defaultprefix

I was able to ignore this for a while until I installed the Custom Hostnaming Extension for vRA7 by Dailyypervisor. This extension is really awesome and has helped me massively overcome my requirement to create dynamically generated hostnames. The hostnames are based on our internal naming convention and I make good use of Custom Properties for this. For testing, I simply created a very basic custom hostname scheme ‘{LOC}-GSTEST{##}’, with {LOC} being set on one of the vSphere endpoints.

hostnamescheme

At first this appeared to work just fine, until randomly some machines would be provisioned using a different hostname. The other hostname was based on another machine prefix that I had created previously and was configured as the default for the Business Group, which was ‘ES-VWOI-MANA##’. The image below shows the results of 7 machines provisioned using the custom hostname extension.

machines

Clearly what is happening here is that on each provisioning attempt, it is randomly selecting a different machine prefix. What is also interesting is that both prefixes get incremented. In fact, that’s another issue with having a default one set for the business group (or even within the Blueprint), is that they are ‘allocated’ and incremented even before the EBS is used (which is when the custom hostnaming extension kicks in, at the requested stage, which is still AFTER the allocation). I have had a small look at the code for the workflow and can see that it’s identifying that the custom hostname scheme already exists, but I haven’t yet looked into why it’s setting the wrong one.

I figured the easiest workaround for now would be to simply remove the default machine prefix from the business group and make sure no one accidently sets it. But this is when I realised you can’t do that by simply editing the group via the IaaS interface. To cut a long story short, you have to manually remove this on the database, both the SQL database used by IaaS and the PostGres database used by vRA.

Read more “Remove ‘Default Machine Prefix’ for a Business Group”

Top 5 vRealize Automation Resources to get you started

OK, so everyone loves a top 5 so here is a list of my top 5 resources for learning and deploying vRealize Automation.

1. VMware Hands on Labs.

Take ‘HOL-SDC-1633 vRealize Automation 7: What’s New‘. – Despite the name this is a really in depth tutorial and you will want to complete the entire lab. This will give you a good feel about what vRA is capable of and how you can extend the platform. There are some really good examples of how the event broker is used to integrate with ITSM CMDB software for change control (iTop is used). The lab also dives into vRO and gives you a taste of just how powerful this product really is.

Next, take ‘HOL-SDC-1632 vRealize Automation Advanced: Integration and Extensibility‘. – Yes, this lab is based on vRA 6.2 so is a little older but most of the fundamentals are there and again provides some good examples of how the platform can be extended with vRO. Examples of extensibility with Infoblox IPAM, Puppet Enterprise and NSX (although slightly depreciated) are used.

2. vRealize Automation Reference Architecture

Once you have had some experience and insight from doing the hands on Labs you will be eager to begin planning and designing your new vRA platform. This document will provide you with a lot of details such as all of the components that are involved and how best to deploy and scale these. Also included are firewall and load balancing requirements. I cannot emphasis enough the importance of planning your vRA deployment properly from the get go as this will ultimately determine the success of the project.

3. Open902.com

I am really happy that I discovered this site before starting my vRA 7 implementation. Michael Rudloff has done a fantastic job of documenting the enterprise installation and configuring the IaaS platform so that you get some decent functionality out of it. These guides really took away a lot of the pain during the installation and covers topics such as replacing certificates, configuring an endpoint, approval policies, business groups, fabric groups, etc and has an awesome guide on Custom Property Relationships. I also like how has turned his private archive public and reminds me a lot of my private Confluence site.

Read more “Top 5 vRealize Automation Resources to get you started”