Advanced vRA Data Collection vRO Workflow

Data Collection in vRealize Automation is a process that keeps the vRA database in sync with endpoints such as vCenter Server. Most people reading this most likely already know how clunky this process is. Also, It’s not immediately obvious how this process can be automated from a vRO prospective. If you are performing any tasks against objects on your endpoints, such as day 2 actions that add or remove hard disks, networks, etc (and not using the OOTB reconfigure actions) then you almost certainly need to be able to programmatically invoke the data collection process to update all the information in vRA.

I have come across many different snippets of code that solved parts of the problem but were far from perfect and the code was ugly. I therefore decided to create something to make this process easy, whilst providing some additional quality of life features. So I present my Advanced Data Collection workflow for your pleasure.

Read more “Advanced vRA Data Collection vRO Workflow”

Resolve vRA/vCAC VM to vCenter VM

I wrote this post some time ago but I felt that it didn’t include some other use cases where you would want to resolve a vCenter virtual machine. I also previously used code that was provided by the vRO appliance that I wasn’t too keen on (logging was a bit light and there was code to search using the BIOS UUID, which we don’t care about). I have therefore updated this post to reflect the following use cases:

  • Find the vCenter VM by its instance Uuid when using the Event Broker;
  • Find the vCenter VM by its instance Uuid on a specific vCenter Server;
  • Find the vCenter VM by its name, on a specific vCenter Server and a specific folder;
  • Find the vCenter VM by its MoRef id on a specific vCenter Server;

Hopefully, there should be enough code provided here to help you achieve your goal no matter what your scenario. However, if you need something different, then feel free to contact me and I will try to help. Read more “Resolve vRA/vCAC VM to vCenter VM”

vRA Developer: Part 3 – Performing CRUD Operations on IaaS Entity Objects

In a previous post vRA Developer: Part 1 – IaaS & Understanding The Entity Framework I detailed what the Entity Framework is and how objects and their properties could be discovered using an application called LINQPad.

This post is going to detail how we can interact with the IaaS servers and work with the objects within the Entity Framework. I have also written a number of actions that provide a standard interface to use when working with the entities and perform CRUD (Create, Read, Update, Delete) operations on them. 

Note: these actions are designed to be used with wrappers for your code (i.e. other actions performing a task, which call these actions) and not directly in your workflows (although there is no real harm if you want to, it’s just cleaner not to). Read more “vRA Developer: Part 3 – Performing CRUD Operations on IaaS Entity Objects”

vRA Developer: Part 2 – Dynamically Discover vCAC and vCACCAFE endpoints in vRO

When I work with endpoints in vRO, I like to use actions to discover these instead of hard coding them as attributes or within configuration elements. Hard coding of these endpoints requires manual configuration steps when moving code between environments, which is not ideal. I felt it would be a good idea to include this post early on within this series, as these hosts are often dependencies when working with a lot of my code.

All code that I have provided or talked about in this post can be downloaded as a vRO package for your consumption here.

Discover vRealize Automation Infrastructure (vCAC IaaS) Endpoint

You will generally only have a single vCAC IaaS endpoint in your vRO inventory. I have never worked in an environment where there was more than one. Therefore, the action I have created below requires no inputs and will attempt to discover this single vCAC IaaS endpoint. If you have more than one host for some reason then it should be fairly easy to incorporate this.

/*global System Server*/

/**
 * Searches the vRO inventory for any instance of a vCACHost object and returns it.
 * Note that this action assumes that only a single vCACHost is present (as is usually the case).
 * @author Gavin Stephens <gavin.stephens@simplygeek.co.uk>
 * @version 1.1.0
 * @function getVcacHost
 * @returns {vCAC:vCACHost} returns the vCAC host object.
 */

var logType = "Action";
var logName = "getVcacHost";
var Logger = System.getModule("com.simplygeek.library.util").logger(logType, logName);
var log = new Logger(logType, logName);
var vcacHosts = [];
var vcacHost;

try {
    log.log("Locating vCAC IaaS Host...");
    vcacHosts = Server.findAllForType("VCAC:VCACHost");
    if (vcacHosts.length === 1) {
        vcacHost = vcacHosts[0];
        log.log("Found vCAC IaaS host: " + vcacHost.name + " [" + vcacHost.url + "]");
    } else if (vcacHosts.length > 1) {
        log.error("More than 1 vCAC IaaS host was found, which this action does not support.");
    } else {
        log.error("Unable to find vCAC IaaS host from the vRO plugins.");
    }
} catch (e) {
    log.error("Failed to locate vCAC IaaS Host.",e);
}

return vcacHost;

Log output:

[] [I] [Action: getVcacHost] Locating vCAC IaaS Host...
[] [I] [Action: getVcacHost] Found vCAC IaaS host: IaaS host for Default [https://web.vra.sgroot.local/]

Read more “vRA Developer: Part 2 – Dynamically Discover vCAC and vCACCAFE endpoints in vRO”

Upgrading External vRO Appliance to 7.5 (for vRA 7.5)

Oh what a joy of an upgrade this one was. Not that it was too difficult, just a tad annoying that an inline upgrade was not possible (or maybe just not supported?, I dunno) and the only way to upgrade to vRealize Orchestrator 7.5 was through a new deployment of the external appliance and migrating the content and settings across. Anyway, I thought it might be useful to others to document the process that I followed to achieve this (and as a future reminder to myself).

Before you begin

This process documents the steps to upgrade a single, standalone appliance but the steps are almost identical for a cluster. If you are using a cluster, then ensure the load balancer is disabled and that the databases are in sync before proceeding. You are responsible for ensuring that all your appliances, virtual machines, databases, etc are backed up and/or snapshot before attempting to perform the upgrade.

Make sure to also snapshot the target vRO server! If anything goes wrong during the migration, then you will need to revert to this snapshot before you can attempt the migration again. Read more “Upgrading External vRO Appliance to 7.5 (for vRA 7.5)”

vRA Developer: Part 1 – IaaS & Understanding the Entity Framework

vRealize Automation 7.x is currently in a sort of ‘split brain’ where there exists two data models which can be used to interact with vRA objects. One is objects that are backed by the Cafe appliance / PostgreSQL database and the other which uses the older, entity framework (IaaS servers).

This post is going to focus on the entity framework, which is still very relevant when working with this version of vRA. There are many things that still do not exist in the newer data model, such as custom properties and data collection. I still see vRO/vRA developers struggle with this, so I hope to help improve the situation.

The Entity Framework

When I first worked with vRA, I struggled to understand how objects were stored and manipulated in the database. I often came across a common object class called an entity. I later discovered that all objects stored in the vRA database are considered ‘entities‘. This is because vRA has been developed with Microsoft’s “Entity Framework“. A brief description of this taken from http://www.entityframeworktutorial.net

The Microsoft ADO.NET Entity Framework is an Object/Relational Mapping (ORM) framework that enables developers to work with relational data as domain-specific objects, eliminating the need for most of the data access plumbing code that developers usually need to write. Using the Entity Framework, developers issue queries using LINQ, then retrieve and manipulate data as strongly typed objects. The Entity Framework’s ORM implementation provides services like change tracking, identity resolution, lazy loading, and query translation so that developers can focus on their application-specific business logic rather than the data access fundamentals.

Entity framework is an Object/Relational Mapping (O/RM) framework. It is an enhancement to ADO.NET that gives developers an automated mechanism for accessing & storing the data in the database.

The ‘domain-specific objects‘ reference is key here and are defined as:

Domain objects are represented by entities and value objects that exist within a domain layer. These objects contribute to a common model and are exposed as a data service, which is also provided by the entity framework.

The entity framework is a layer of abstraction that sits on top of the underlying relational database (SQL Server). This abstraction allows developers to work within a standard framework. Yes, you could run SQL queries on the underlying database directly, but this gets really ugly and isn’t supported.

LINQ is Microsoft’s .NET Language-Integrated Query (language).

What is also important to note is that entities follow some form of continuity and identity (i.e. they must all have certain attributes, such as an ID field or callable methods). This standard allows for a consistent interaction with the domain objects.

In the case of vRA, all domain-level objects (entities) are provided under the ‘ManagementModelEntities.svc‘ data service model. Within this data service model, entities are organised into their own ‘tables’, known as ‘Entity Sets’, and entities can also link to (relate to) other entities. Getting an understanding of the entities will make your life as a vRA/vRO developer so much easier. Read more “vRA Developer: Part 1 – IaaS & Understanding the Entity Framework”

Using the indexOf() method for Arrays and Strings for vRO and vRA

I’ve never been a developer so getting into JavaScript was quite a challenge at first and I probably always went the longest route possible to achieve something. As I use it more and more, I am picking up these neat little tricks and uses for built in methods that make my life easier.

In the world of vRA and vRO, I find that most of my time is spent iterating over arrays or parsing custom properties. One method that I have come to find extremely useful is the indexOf() that is available on Arrays and Strings. The methods are very similar but have very different use cases. Let’s take a look at each of them in turn.

String indexOf() Method

w3schools.com defines this as:

The indexOf() method returns the position of the first occurrence of a specified value in a string.

This method returns -1 if the value to search for never occurs.

So as an example, if we had the string “simplygeek.co.uk is fun”

string.indexOf(“m”) would return 2, which is the index within the string that ‘m’ first appears. Note that if ‘m’ appeared twice then only the first match would return a result. Indexes within arrays and strings always start at 0.

Another use case, one which I find the most useful, is being able to provide a string for the lookup. Take the following example:

string.indexOf(“simplygeek”) would return 0, because in a contiguous match the first index is returned.

When writing JavaScript that interacts with vRA you are often required to parse through custom properties, which are key value pairs of data. Such properties can contain useful information that relates to a deployment, such as virtual machine configuration. If custom properties follow a standardised naming convention, it can be easy to discover a set of properties. Let’s assume I have created the following custom properties in vRA for a deployment:

Custom.Deployment.Virtualmachine.Config.hotcpu : true
Custom.Deployment.Virtualmachine.Config.hotmem : true
Custom.Deployment.Virtualmachine.Config.sched.swap.vmxSwapEnabled : true

When the payload is sent to my vRO workflow it could contain over a 100 different key:value pairs of data. To find these easily I can use the indexOf method to iterate over each pair as follows:

var deploymentProp = new Properties;
var arrayOfDeploymentProps = [];
for each (var key in props.keys) {
    if (key.indexOf("Custom.Deployment.Virtualmachine.Config") === 0) {
        // We found a deployment property!
        deploymentProp.put(key, props.get(key));
        arrayOfDeploymentProps.push(deploymentProp);
    }
}

The above will result in an array of properties related to virtual machine configs. I can then pass this array to some code that will handle the implementation of these advanced virtual machine settings. This allows for a very dynamic way to manage custom properties in property groups within vRA.

Array indexOf() method

Very similar to the String method, on an array, the indexOf() method returns the first index at which a given element can be found in the array, or -1 if it is not present. I find this method useful when I need to return a set of unique values from another array. let’s assume we have the following array:

myArray = [‘one’, ‘two’, ‘two’, ‘three’, ‘three’, ‘three’]

If I wanted to return only unique items from myArray, I could use the indexOf method as follows:

var myArray = ['one', 'two', 'two', 'three', 'three', 'three'];
var uniqueItems = [];
for each (var item in myArray) {
    if (uniqueItems.indexOf(item) === -1) {
        // we didn't find this item in the array so add it.
        uniqueItems.push(item);
        // When we get to the second 'two', the value of indexOf() will return '2' and the code above will not be executed.
    }
}

The above code will result in an array:

[‘one’, ‘two’, ‘three’]

I hope that someone else finds these as useful as I have. If you know of more use cases, then please let me know.

vRA 7 – Getting more information in workflows from the vRO ExecutionContext object

When a vRO workflow or action is called from vRA, additional input parameters (in addition to those specified as the workflow/action inputs) are provided in the Execution Context object of the workflow. These can be very useful as they contain additional data that can be used inside the workflows. A couple of good examples would be the user that requested the resource or the name of the tenant.

All code that I have provided or talked about in this post can be downloaded as a vRO package for your consumption here.

Parameters Available

Here is a list of parameters provided inside the Execution Context (those without a description I’m still trying to figure out) of a workflow that has been executed by the Event Broker during VM provisioning.

 Parameter Name  Description
__asd_catalogRequestId The Request ID (vCAC Entity object reference)
__asd_correlationId  The Request ID (vCACCAFE object reference)
__asd_requestInstanceId
__asd_requestInstanceTimestamp Request Date & Time (UTC)
__asd_requestInstanceTypeId The lifecycle event topic id i.e. com.vmware.csp.iaas.blueprint.service.machine.lifecycle.provision
__asd_requestTraceId
__asd_requestedBy The UPN of the user who made the request
__asd_requestedFor The UPN of the user the request was on behalf of
__asd_targetResourceId The vCAC virtual machine entity id
__asd_targetResourceTypeId Type ID, i.e. ‘machine’
__asd_targetResourceProviderId
__asd_targetResourceProviderTypeId com.vmware.csp.iaas.blueprint.service
__asd_tenantRef The id of the tenant (same id which goes in the url)

If anyone can shed some light on what the other parameters do then please comment and I’ll update the page.

You can also see these in the variables tab for the workflow run:

There is also another parameter which does not appear:

 Parameter Name  Description
__asd_subtenantRef The business Group id

This parameter is used to identify the business group which the user who submitted the request is a member of. For some reason this isn’t made available like the above parameters but only becomes available when you create and submit an XaaS Blueprint (a catalogue request mapped directly to a workflow).

The only workaround that I have used to get Business Group information available in Event Broker subscriptions is to create a custom property on each business group called ‘Custom.Business.Group.Name’ with the string name of the business group. It’s not a great solution but it does allow the custom property to be queried in the usual way.

Retrieve Parameter Values

There are two ways that you can retrieve the values from these parameters. The first is to simply add these parameters as an input to the workflow. vRA will automatically detect and populate these parameters with the corresponding values. You can then use these inputs as you normally would in your workflow.

The second option is to retrieve them from the Execution Context object.

Workflow Execution Context

The parameters and their values in the execution context object can be retrieved using the System scripting class. There is a method called ‘getContext()‘, which returns an object called ‘ExecutionContext‘. The ExecutionContext object has a ‘getParameter’ method which can be used to retrieve the value for the specified parameter.

Insert the following code into a scripting task in your workflow to list all the parameters and their values:

var executionContext = System.getContext();

System.log("\nExecution context Parameters:"
for each (var parameter in executionContext.parameterNames().sort()) {
    System.log(parameter + " : " + executionContext.getParameter(parameter))
}

Retrieve the Value for a Specific Parameter

To retrieve a value for a specified parameter the following example can be used:

Retrieves the tenant identifier and sets it to a variable ‘tenant’.

var tenant = System.getContext().getParameter("__asd_tenantRef");

Retrieve Parameter Values using Actions

I always like to do everything using Actions as I like to write my code once and simply re-use them repeatedly by dropping them onto my workflows. I have created a collection of Actions that can be used to get values from the Execution Context. As always, a link is provided to all my code at the beginning of this page.

I have a ‘core’ helper Action that will do the main work of retrieving the parameter value. This is a generic Action for getting a value from the Execution Context and is not directly related to vRA.

getExecutionContextParameterValue

/*global parameter*/

/**
 * Retrieves the value for the specified parameter in the Execution Context.
 * @author Gavin Stephens <gavin.stephens@simplygeek.co.uk>
 * @version 1.2.0
 * @function getExecutionContextParameterValue
 * @param {string} parameter - The Execution Context parameter.
 * @returns {string} Returns the parameter value.
 */

function checkParams(parameter) {
    var inputErrors = [];
    var errorMessage;
    if (!parameter || typeof parameter !== "string") {
        inputErrors.push(" - parameter missing or not of type 'string'");
    }
    if (inputErrors.length > 0) {
        errorMessage = "Mandatory parameters not satisfied:\n" + inputErrors.join("\n");
        log.error(errorMessage);
    }
}

var logType = "Action";
var logName = "getExecutionContextParameterValue"; // This must be set to the name of the action
var Logger = System.getModule("com.simplygeek.library.util").logger(logType, logName);
var log = new Logger(logType, logName);
var paramValue = "";

try {
    checkParams(parameter);
    log.debug("Getting value for parameter '" + parameter + "'");
    paramValue = System.getContext().getParameter(parameter);
    if (paramValue) {
        log.debug("Found parameter value: " + paramValue);
    } else {
        log.error("Could not find value for parameter '" + parameter + "'");
    }
} catch (e) {
    log.error("Action failed to get parameter value.",e);
}

return paramValue;

The following Action is a wrapper for the ‘getExecutionContextParameterValue‘, that is used to get a value for a specific parameter.

getTenantRefParameterValue

/**
 * Returns the value for the parameter '__asd_tenantRef'.
 * @author Gavin Stephens <gavin.stephens@simplygeek.co.uk>
 * @version 1.1.0
 * @function getTenantRefParameterValue
 * @returns {string} Returns the parameter value.
 */

var logType = "Action";
var logName = "getTenantRefParameterValue"; // This must be set to the name of the action
var Logger = System.getModule("com.simplygeek.library.util").logger(logType, logName);
var log = new Logger(logType, logName);
var parameter = "__asd_tenantRef";
var paramValue;

try {
    log.log("Getting value for parameter '" + parameter + "'");
    paramValue = System.getModule("com.simplygeek.library.vro.workflow").getExecutionContextParameterValue(parameter);
    if (!paramValue) {
        log.error("No value was found for parameter '" + parameter + "'");
    }
    log.log("Found parameter value '" + paramValue + "'");
} catch (e) {
    log.error("Action failed to get value for parameter '" + parameter + "'",e);
}

return paramValue;
[2019-01-31 22:47:07.909] [I] [Action: getTenantRefParameterValue] Getting value for parameter '__asd_tenantRef'
[2019-01-31 22:47:07.926] [D] [Action: getExecutionContextParameterValue] Getting value for parameter '__asd_tenantRef'
[2019-01-31 22:47:07.929] [D] [Action: getExecutionContextParameterValue] Found parameter value: sg
[2019-01-31 22:47:07.934] [I] [Action: getTenantRefParameterValue] Found parameter value 'sg'

You would simply drop this Action on to your workflow and set the output to store the parameter value.

I have included all the above Actions plus those required to retrieve all parameter values.

As always, if you need any help then please drop me a message via the Drift app.