Deploying vRealize Build Tools To Allow Infrastructure As Code for vRA and vRO

As any vRealize Orchestrator developer will tell you, managing code outside of the appliance is difficult. In fact, I recently wrote a post about Using Visual Studio Code for your vRealize Orchestrator Development, where I highlighted some of the challenges with this. The issue is that we’re not really given the freedom to use any IDE we want, easily run unit tests on our code or do continuous integration with tools like Jenkins.

I did mention that a couple of solutions were going to make their way, one of these was internal tooling that VMware’s CoE team currently use for their own vRO development (you can read the article here: https://blogs.vmware.com/management/2018/11/automating-at-scale-with-vro-and-vra.html). It wasn’t possible to get access to these tools without engaging with CoE and forking up a bit of cash.

That is until now, as VMware have released these tools as a new fling. The fling is currently in preview, but you can check it out here: https://labs.vmware.com/flings/vrealize-build-tools. I think this is quite an exciting time for VMware developers as these tools could finally change the way we develop and manage our code and integrate into the wider developer ecosystem.

This is my first blog on this topic but if I find these tools useful, then there will be plenty more to follow. Getting the environment set up to use these tools is not straight forward and has a number of dependencies. These include deploying supporting infrastructure such as JFrog Artifactory, preparing all the required artifacts that are sourced from the vRO server and getting the workstation set up to create and manage packages.

Deploy and Configure Platform

Before the developer can begin using the vRealize Build Tools, the supporting platform has to be deployed and configured. This consists of setting up an Artifactory server to store Maven artifacts and build dependencies and preparing the artifact repositories.

Deploy Artifactory Server (skip if you already have this deployed)

This section will detail how to set up the Artifactory server and required dependencies. Note that the details below only deploy a single Artifactory node with the database instance running on the same machine. It is recommended that for a production environment to ensure Artificatory is deployed with high availability and connects to external/dedicated database instances.

Install Java Development Kit (JDK)

JFrog Artifactory requires the Java JDK 8 and above to be installed and the JAVA_HOME variable configured. I am using the Open Source version of these tools. Install using the following command:

Add the following lines to ‘/etc/profile‘ to set the ‘JAVA_HOME environment variable and add the Java bin directory to the path.

Then source the file and check that the variables have been correctly configured:

Install Maria DB

Artifactory requires MySQL or a compatible version to be installed so we’re going to go with MariaDB, which appears to be widely popular with Artifactory installations. You can install this directly from the CentOS repositories, but you may need to add additional repositories for other flavours.

Start and enable the Maria DB instance:

Next, you need to configure the Maria DB instance by running the following script which will perform a secure setup:

This will ask you to enter the current root password (just press enter) and set a new password. You can access the default responses for all other questions. Once this is complete, the Maria DB instance is ready.

Install Jfrog Artifactory

We will need to install the Open Source (OSS) version of Artifactory, which provides support for Maven. First add the required repository using curl:

And next install Artifactory OSS

Set the ‘ARTIFACTORY_HOME‘ variable in ‘/etc/profile‘ to the location of the Artifactory installation and source the file:

Next, configure Artifactory to use the Maria DB database:

Provide the root username and password for Maria DB and set the username and password used by Artifactory to connect to the database. If the server has access to the Internet, it will download the required MySQL connector.

Finally, start the Artifactory service:

And the bit that caught me out… enable exceptions to access this service in the firewall:

You should now be able to access the Artifactory page at http://artifactory_host:8081

Configure Jfrog Artifactory

There isn’t much to configure but run through the configuration wizard and set a new admin password and proxy (if required). At the Create Repositories page, select Maven.

The base configuration is now complete and the following Maven repositories will have been created:

  •  libs-snapshot-local
  •  libs-release-local
  •  jcenter
  •  libs-snapshot
  •  libs-release

Generate API Access Key

So that we’re not passing plain text usernames and passwords when working with Artifactory, it’s best to create an API Key.

Log into the Artifactory GUI and select your username in the top right and when the drop down appears, select ‘Edit Profile‘.

You will be asked to enter your password to unlock these settings. Once you have done so, click on the cog iconto generate an API key for your user. Copy this so that it’s easily accessible for use later.

Install Jfrog CLI

You will need to use the jFrog CLI to upload required artifacts to the repositories. You can download this single binary using curl and then copy it to a location in the environment path.

Check that the CLI is working properly by outputting the version:

The platform is now ready to be used with vRealize Build Tools.

Create vRealize Artifact Repositories

This section will cover getting the Artifactory repositories set up and preparing and uploading the vRealize artifacts.

Create vRealize Repository in Artifactory

Before we begin to prepare the vRealize artifacts, we need to create a new local repository in Artifactory. Log into the Artifactory GUI and navigate to the ‘Admin‘ section. Under ‘Repositories‘ select ‘Local‘.

On the ‘Local Repositories‘ page click the ‘New’ button in the top right.

When asked to ‘Select Package Type‘, select Maven.

Name the repository (Repository Key) vRealize-local.

Leave all other settings as-as and click Save & Finish.

You will now need to add this new local repository to the virtual release repository libs-release.

Navigate to the ‘Admin‘ section. Under ‘Repositories‘ select ‘Virtual‘.

From the list of repositories, select libs-release. On the right, you will see Repositories that are Available and those which have been Selected. Select the vRealize-local repository and click the single right arrow button to move it into the Selected section.

To complete, click Save & Finish.

Artifactory is now ready to store vRealize artifacts.

Prepare vRealize Artifacts

I am going to prepare all the vRealize artifacts locally on the Artifactory server, but you can also do this on any server that you wish to use for CI.

If you haven’t already done so, download the vRealize Build Tools package and copy it to your home directory on the CI/Artifactory server directory (You may have to SCP the file).

You will also need to install the unzip command:

Create a new directory to store the contents and unpack the file:

Next, navigate to the artifacts directory that has been created and verify that you can see the ‘iac-maven-repository.zip‘ file:

Create the following directory (in your home directory) which will store all the vRealize artifacts.

Move the ‘iac-maven-repository.zip‘ file to this new directory:

Next, switch into this directory and use the unzip command to unpack the file:

And finally, delete the ‘iac-maven-repository.zip‘ file.

Confirm that the directory now contains a com directory and the archetype-catalog.xml file.

Modify POM Files For vRO Version

The vRealize Build Tools has a requirement to stand up a vRO 7.3 appliance. However, I have removed this requirement and detailed how to obtain the required vRO artifacts from the version of your choice (or a later version than 7.3!). This is covered in the Prepare vRealize Orchestrator Artifacts section.

The vRealize artifacts contain a number of POM files where the vRO version 7.3 version is hard coded. We will need to change this to match the vRO version that you are going to retrieve the artifacts from.

The easiest way to identify where this change is required is to run the following command:

This command is going to recursively match the pattern ‘7.3.0’ and return the file where it was found, including the line number.

You will need to modify the following POM files:

  • ~/vrealize_artifacts/com/vmware/pscoe/o11n/o11n-convert/1.5.11/o11n-convert-1.5.11.pom (1 occurrence)
  • ~/vrealize_artifacts/com/vmware/pscoe/o11n/hint-collector/1.5.11/hint-collector-1.5.11.pom (3 occurrences)
  • ~/vrealize_artifacts/com/vmware/pscoe/o11n/o11n-project/1.5.11/o11n-project-1.5.11.pom (1 occurrence)
  • ~/vrealize_artifacts/com/vmware/pscoe/o11n/xml-package/1.5.11/xml-package-1.5.11.pom (1 occurrence)
  • ~/vrealize_artifacts/com/vmware/pscoe/o11n/o11n-jasmine/1.5.11/o11n-jasmine-1.5.11.pom (1 occurrence)
  • ~/vrealize_artifacts/com/vmware/pscoe/o11n/o11n-local/1.5.11/o11n-local-1.5.11.pom (1 occurrence)

I am using vRealize Orchestrator 7.5, so naturally, I am going to set this value to 7.5.0.

Upload vRealize Artifacts

Now with the vRealize artifacts prepared, they need to be uploaded to the vRealize-local repository that was created in Artifactory. To do this, you can use the Jfrog CLI by specifying the Artifactory URL, the API key created earlier for your account, the local directory to upload and the target Artifactory repository.

If you get asked to configure the CLI then select ‘N‘ as we’ll be specifying the URL and API at run time (there is a bug in the config when using an API key).

You will see the uploads begin and once this has finished you’ll be presented with a summary and hopefully that the upload was successful.

You have completed the preparation of the vRealize artifacts.

Create vRealize Orchestrator Repository in Artifactory

Create a new local repository in Artifactory called vro-local and add it to the virtual release repository lib-release (follow the same steps from the previous section).

Prepare vRealize Orchestrator Artifacts

Create the following directory (in your home directory) which will store all the vRealize Orchestrator artifacts and then switch to this directory.

The documentation provided with the vRealize Build Tools asks for artifacts to be pulled from a vRO 7.3 appliance. I noticed that these repositories are exposed through a web service and can easily be pulled using wget. However, this does not appear to be the same case for later versions of vRO (I’m using vRO 7.5). I did a little digging and discovered that these repositories do actually exist on the vRO appliance. Therefore, I decided to take the artifacts from my 7.5 appliance and use SSH/SCP to copy the required files across.

Upload vRealize Orchestrator Artifacts

Now with the vRealize Orchestrator artifacts prepared, they need to be uploaded to the vro-local repository that was created in Artifactory. To do this, you can use the Jfrog CLI by specifying the Artifactory URL, the API key created earlier for your account, the local directory to upload and the target Artifactory repository.

You will see the uploads begin and once this has finished you’ll be presented with a summary and hopefully that the upload was successful.

Configure Anonymous User Permissions

The Artifactory anonymous user will need to be granted permissions to read and deploy from the local repositories. This allows the build clients to access these repositories anonymously when creating and building packages.

Log into the Artifactory GUI and navigate to the ‘Admin‘ section. Under ‘Security‘ select ‘Permissions‘.

On the Permissions Management screen, click New in the top right.

Name the permission ‘Anonymous Cache‘.

Under the Resources tab, we need to add all of the Available Repositories to the Included Repositories section using arrow button.

Under the Users tab, the anonymous user needs to be selected and granted the following permissions:

  • Read, Annotate, Deploy / Cache

To complete, click Save & Finish.

Install vRealize Orchestrator Hints Plugin

As part of the vRealize Build Tools there is a vRO plugin provided in the ‘artifacts\vscode‘ directory in the downloaded package (tar.gz file). This plugin needs to be installed on the target vRO server and allows Visual Studio Code to discover the plugins and provide IntelliSense and code completion.

Log into the vRO Control Centre and install the ‘o11nplugin-hint-1.5.11.dar‘ plugin.

The vRO service will need to be restarted to complete the installation.

Make sure that you are able to connect to the vRO server using the client before continuing further.

Prepare and Configure Developer Workstation (Windows)

In order to begin using the vRealize Build Tools, the developer workstation has to be prepared with the required prerequisites met. Note that I am preparing a Windows 10 workstation so if you are using a Mac or Linux, then you may have to seek alternative guidelines on completing these steps.

Prepare Developer Workstation

The following sections will detail the software requirements and configuration that is needed to allow your workstation to build packages for vRO.

Install Java Development Kit (JDK)

Maven will only work with Java JDK installed (not Java Runtime Environment (JRE)) but the JRE is also included. You will need to install Java JDK version 8, which can be downloaded directly from Oracle: https://www.oracle.com/technetwork/java/javase/downloads/index.html.

Once this has been installed, you will need to set the JAVA_HOME environment variable to point to the installation path of the JDK.

Open a command prompt and issue the following command, replacing the JDK path with your own.

You will also need to add the JDK bin directory to the system path.

Open a command prompt as an Administrator and issue the following command.

Install Apache Maven

Apache Maven is used to manage and build projects, based on the Project Object Model (POM). Download the Maven binary package from https://maven.apache.org/download.cgi.

The binary package does not include an installer so extract to a directory, that does not included any spaces. (I have mine at D:\apache-maven\3.6.1).

Add the maven bin directory to the environment path. Open a command prompt as an Administrator and issue the following command, replacing the maven bin path with your own.

Verify that you can execute Maven by displaying the version.

Create Project Directory

Create a project directory that will be used to store all the development files. I typically have a projects folder in my users’ home directory and have created a folder called vRO-vRA within it i.e. ‘C:\Users\Gavin\projects\vRO-vRA\’.

Create Java Keystore

All packages need to be signed during build time, using a Java Keystore file. Open a command prompt and navigate to the root of your project directory (not where the code will live). Issue the following command to create the Java keystore file, replacing with your own details:

Create Master Password

Maven allows encrypted server passwords to be stored in the settings.xml file. Before Maven can do this, it requires that a master password first be set. This is achieved using the ‘mvn –encrypt-master-password‘ command, that will prompt you to enter the password that will be encrypted:

You will need to store this encrypted string in the ‘%USERPROFILE%/.m2/settings-security.xml‘ file. If this does not already exist, then create it as follows:

Configure Project Settings (settings.xml)

You will need to create a ‘%USERPROFILE%/.m2/settings.xml‘ file that is based on the below template (taken directly from the vRealize Build Tools documentation, with a correction to the vro.auth tag), ensuring to replace all the {} placeholders with your own values.

The ‘{native+maven+encrypted+pass}‘ is the encrypted password that is generated using the ‘mvn –encrypt-password‘ command.

Install Visual Studio Code

Now we’re on to the good stuff. If you don’t already have it installed, then go ahead and install Visual Studio Code. My post on Using Visual Studio Code for your vRealize Orchestrator Development can help you out here if you need an overview or guidance.

Install vRealize Developer Tools Extension

The next step is to install the vRealize Developer Tools extension. With Visual Studio Code open, click on the extensions icon and search for vRealize Developer Tools and install the extension.

Make sure to restart Visual Studio Code after the extension has been installed as I had issues when I didn’t do this.

Next, open the project folder for your vRO code (if you recall earlier we created a projects folder and a sub folder to hold all your vRO code). I use the project folder ‘%USERPROFILE%\projects\vRO-vRA‘.

Using The vRealize Build Tools

This section will detail how you can begin creating new projects, load your maven profile and push content to the vRO server.

Create Your First Project

The extension won’t activate at first, because the project folder is currently empty. Start by creating a new vRealize project by pressing CTRL+SHIFT+P and search for the command vRealize: New Project.

Select the ‘vRO JavaScript based‘ project.

Provide a group ID for the project.

Finally, provide a name for the project. The group ID and the name combined form the module name that you would see in the vRO client under Actions.

You will then be prompted where to create this new project. It should default to your project folder so just click ‘Create Here‘.

You may encounter an error at this stage if you are running Visual Studio Code in Windows.

There appears to be a bug in the command string that will prevent the project from being created. If you also run into this problem then you will have to create the project on the command line using Maven. You can simply copy the command that is provided in the error.

Then just open a command prompt to your project folder and execute the command. You will see all the artifacts being downloaded from Artifactory and hopefully that the build completed successfully:

Load Profile and Download Hints

Switch back to Visual Studio Code and you should now see that the vRealize Developer Tools extension has loaded for this project. This can be identified from the profiles icon in the bottom left side of the IDE

Click on this icon and select the profile you would like to work with (this is the profile that was configured in the settings.xml file).

The status bar in the bottom left will change and your selected profile will be displayed. A cloud icon will also now appear. This is used to trigger a hints collection against the vRO server. If you recall earlier, you installed the hints plugin on the vRO server. This will download information about all the installed plugins and vRO intrinsic functions.

Click the hints icon to invoke a hints collection.

In the Output window in Visual Studio Code for vRO – Language Server, you will see the hints collection being processed.

You will now get IntelliSense when developing code for vRO and using plugins.

Push Package to vRO

Perform this task on the first project that you created as a test to ensure that you are able to promote packages to the vRO server. From the command line, move into the new project directory and execute the following command:

Where ‘test‘ is the name of the project created earlier and ‘development‘ is your profile name.

Providing that everything has been set up correctly, the build should be successful:

And the package should now be visible in vRO, with a sample action in the correct module.

If you got this far and everything has worked then well done. There is still much more to cover and I’ll be working on a number of followup posts exploring more than can be achieved using these build tools. I’ll also look at how we can effectively manage projects and code and look at unit testing using Jasmine.

Thanks for reading and I hope that this post has been useful. If you have any problems getting these tools to work, then please drop me a message and I will do what I can to help.

Please rate this post!

Average rating / 5. Vote count:

Leave a Reply

avatar
  Subscribe  
Notify of