CentOS running VirtualBox (headless mode)

Today I started work on something that has peeked my interest for a while, switching my server to CentOS running VirtualBox in headless mode.

I had long been a fan of VMware Workstation, back in the day, it was more feature rich than Vbox and provided better memory management features. Alas, that is not the case today and with VirtualBox’s powerful ‘VBoxManage‘ CLI, it really fits in well where I can write all my infrastructure in code (and yes, I’ll most likely layer Vagrant on top of this). VirtualBox also provides an alternative headless interface ‘VBoxHeadless‘, which means there is no requirement to run a GUI on my server.

As I am starting out on this new journey I felt that it would be great to blog about and hopefully help others that want to do this.

All I have from the start is a minimal installation of CentOS 7.4 (Core). I am using the following as a guide for the VirtualBox installation: https://wiki.centos.org/HowTos/Virtualization/VirtualBox.

VirtualBox Installation

Install Dependencies

Install Extra Packages for Enterprise Linux (EPEL)

Install Dynamic Kernel Module Support (DKMS)

This will install quite a few packages:

Install Development Tools

I want my server to have access to a basic development environment so we’ll install the group packages for this. This will install packages such as gcc, make, binutils, etc. Use ‘yum groupinfo “Development Tools“‘ to view the entire list of packages installed in this group.

Install Kernel Development

Install VirtualBox

Add the VirtualBox package repository

Install VirtualBox

This will also install a number of dependencies.

Install VirtualBox Extension Pack

The VirtualBox Extension Pack will add support for the following:

  • USB 2.0 and USB 3.0 Host Controller
  • Host Webcam
  • VirtualBox RDP
  • PXE ROM
  • Disk Encryption
  • NVMe

Download the extension pack:

Once downloaded, install the extension pack:

OR, if you are upgrading from a previous version of the extension pack, then you will need to add the ‘–replace’ option to uninstall the old version first.

Agree to the license terms and conditions.

The extension will be installed.

Verify that the extension pack has been installed successfully.

You should see output like the following:

Add Users to ‘vboxusers’ Group

When VirtualBox is installed a new group ‘vboxusers’ is created. Users that are a member of this group will be allowed to run VirtualBox. I will add my non-privileged user to this group.

Verify Installation

VirtualBox Linux kernel module (vboxdrv)

Check that the vboxdrv service has installed correctly and is running.

You should see output like the following:

The status should be ‘loaded‘ and ‘active‘. If you see a ‘Kernel driver not installed‘ message then try running the following command:

VirtualBox Web Service (vboxweb)

Check that the vboxweb service is running:

You should see output like the following:

Configure VirtualBox

Configure Networking

I have some basic network requirements for my lab to start out with. I will use a host-only interface, which will be the management network and a NAT interface, which can be useful for VMs that I want to access the Internet without going through the virtual firewall. I will also need to bridge one of my virtual firewall’s network interfaces to the servers physical interface (so that I can do some additional routing on my physical network).

I can see what network interfaces are currently available using nmcli.

Create the Host-Only Interface

We can confirm that the Host-Only interface was successfully created using nmcli.

Assign Network to Host-Only Interface

To confirm that the IP address has been assigned, use the ip command.

Create NAT Interface

I want to create a NAT network on 192.168.15.0/24 and have IP addresses allocated to clients automatically using DHCP.

To confirm that the NAT network has been created use the following command.

We can see that the network has been created and an IP address has been allocated automatically for the gateway (uses the first available address). This should be enough to provide outbound Internet access to any VM NICs attached to this interface.

Global Settings

I also like to configure some global settings so that I do not need to keep specifying these when creating new virtual machines.

Default Virtual Machine Folder

I actually have 2 large SSDs that I spread the VMs across for IO reasons. I will default to one of these and manually specificity my second disk when required.

Exclusive Hardware Virtualization

VirtualBox will be given exclusive use of the hardware virtualization extensions (Intel VT-x or AMD-V). I think this defaults to on but let’s set it anyway.

Default Front End

I am not running a GUI on this server so all virtual machines will be running in headless mode.

That concludes my initial setup and now I am ready to start deploying my virtual machines. I will provide posts on my new virtual machine deployments as I build out my new infrastructure using VirtualBox. I also have a number of OVF’s that I exported from my old environment that I’ll be importing and will document the steps along the way.

Using the indexOf() method for Arrays and Strings for vRO and vRA

I’ve never been a developer so getting into JavaScript was quite a challenge at first and I probably always went the longest route possible to achieve something. As I use it more and more, I am picking up these neat little tricks and uses for built in methods that make my life easier.

In the world of vRA and vRO, I find that most of my time is spent iterating over arrays or parsing custom properties. One method that I have come to find extremely useful is the indexOf() that is available on Arrays and Strings. The methods are very similar but have very different use cases. Let’s take a look at each of them in turn.

String indexOf() Method

w3schools.com defines this as:

The indexOf() method returns the position of the first occurrence of a specified value in a string.

This method returns -1 if the value to search for never occurs.

So as an example, if we had the string “simplygeek.co.uk is fun”

string.indexOf(“m”) would return 2, which is the index within the string that ‘m’ first appears. Note that if ‘m’ appeared twice then only the first match would return a result. Indexes within arrays and strings always start at 0.

Another use case, one which I find the most useful, is being able to provide a string for the lookup. Take the following example:

string.indexOf(“simplygeek”) would return 0, because in a contiguous match the first index is returned.

When writing JavaScript that interacts with vRA you are often required to parse through custom properties, which are key value pairs of data. Such properties can contain useful information that relates to a deployment, such as virtual machine configuration. If custom properties follow a standardised naming convention, it can be easy to discover a set of properties. Let’s assume I have created the following custom properties in vRA for a deployment:

Custom.Deployment.Virtualmachine.Config.hotcpu : true
Custom.Deployment.Virtualmachine.Config.hotmem : true
Custom.Deployment.Virtualmachine.Config.sched.swap.vmxSwapEnabled : true

When the payload is sent to my vRO workflow it could contain over a 100 different key:value pairs of data. To find these easily I can use the indexOf method to iterate over each pair as follows:

The above will result in an array of properties related to virtual machine configs. I can then pass this array to some code that will handle the implementation of these advanced virtual machine settings. This allows for a very dynamic way to manage custom properties in property groups within vRA.

Array indexOf() method

Very similar to the String method, on an array, the indexOf() method returns the first index at which a given element can be found in the array, or -1 if it is not present. I find this method useful when I need to return a set of unique values from another array. let’s assume we have the following array:

myArray = [‘one’, ‘two’, ‘two’, ‘three’, ‘three’, ‘three’]

If I wanted to return only unique items from myArray, I could use the indexOf method as follows:

The above code will result in an array:

[‘one’, ‘two’, ‘three’]

I hope that someone else finds these as useful as I have. If you know of more use cases, then please let me know.

vRO Coding Pet Hate – Badly formed IF statements

I was optimising some vRO actions today and came across something that always annoys me when I see it, badly formed IF statements (for one of a better description). When I write my code, I have learned that formatting and consistency is key to produce code that is easier to read and understand. I also like to keep things simple. However, I am constantly surprised at how much sloppy coding practices I encounter from the code that is shipped with vRO. Here are a couple of small examples of the kind of thing I am talking about.

Both of the above statements are pretty much identical in how they function. They are checking if a given variable is null (or not) and if the condition returns ‘true’ then execute the following statement. As you can see from the code snippets above, the structure of these statements are different. Why? Because this is a valid way to write conditionals in JavaScript, if you are only performing an action when the result is ‘true’. That said, it doesn’t mean we should use it in practice. The reason for this, is that it forces a lack of consistency in your code (which you can already see from the code above).

If we wanted to add another action for a result that returns ‘false’, we would need to add an ‘else’ clause. With the conditional statements above, we wouldn’t be able to do this as it would require adding curly braces so that the interpreter knows when to apply the correct code based on the result. Here is an example.

This is the structure that I like to adhere to every time I write a conditional. I also apply the ‘4 spaces rule’ for my indentation (and don’t get me started on the use of tab!). The two code snippets revised using this structure would look like:

I believe this is a much neater approach for the sake of a few more lines, which aren’t worth saving if the code is more readable.

Get a list of Datastores, quickly, using XPATH

The vCenter plugin has a very useful method for retrieving a list of objects quickly and easily. However, if your inventory is quite large, then this can be quite slow. There are two ways that this can be improved. 1) only retrieve the attributes that are useful for our code requirements, 2) use an XPATH query filter to limit the scope of the search.

In my situation there were over a 100 Datastores but I only wanted to retrieve a list based on a given prefix (of which there were less than 10). The Datastore names end with an incrementing number and I wanted to return all of these in an array.

I have used a similar approach which allowed me to retrieve several thousand virtual machines in approximately 3-5 seconds.

Get Datastore with Most Free Space & Check Datastore Meets Capacity Reservation

I do a lot of work that involves either creating new virtual hard disks or attaching existing ones to virtual machines in VMware vSphere. I do all of this through vRealize Orchestrator, written in JavaScript (yum).

As part of this task, I always ensure that the datastores that I am creating disks on have sufficient capacity for these new disks. I use two simple functions that I have written which perform these checks for me, which is ‘getDatastoreWithMostFreeSpace’ and ‘DoesDSMeetCapcityReserve’. An additional function ‘convertToGB’, which is used by these functions is also provided at the end.

Function : getDatastoreWithMostFreeSpace

Description
Return a single VcDatastore object that has the most free space from an Array of provided VcDatastore objects. The function will always return a VcDatastore, regardless of how much free space is found.

Parameters
– VcDatastores (Array of VcDatastore)

Return Type : VcDatastore

Function : DoesDSMeetCapcityReserve

Description
Checks that a given VcDatastore has enough free space in which to place the virtual disk, taking into account current usage and a defined reservation value (in percent) that should be guaranteed. Returns true if the Datastore passes the capacity checks.

Parameters
– datastore (VcDatastore)
– reservation (Number)
– totalSizeRequired (Number)

Return Type : Boolean

Function : convertToGB

Description
Converts a value in Bytes to GB.

Parameters
– size (Number)

Return Type : Number

Example Output

[2017-09-18 15:09:37.459] [I] Selecting the datastore with the most free space:
[2017-09-18 15:09:37.714] [I] Datastore 1 information:
[2017-09-18 15:09:37.715] [I] name: DATASTORE1
[2017-09-18 15:09:37.717] [I] ID: datastore-99999
[2017-09-18 15:09:37.718] [I] Capacity: 2048 GB
[2017-09-18 15:09:37.719] [I] Free Space: 1762 GB
[2017-09-18 15:09:37.721] [I] Used Space: 286 GB
[2017-09-18 15:09:37.725] [I] MaxVMDKSize: 63488 GB
[2017-09-18 15:09:37.726] [I] Selected Candidate Datastore: DATASTORE1
[2017-09-18 15:09:37.728] [I] Performing capacity validation checks on the datastore.
[2017-09-18 15:09:37.729] [I] Datastore reservation is set to 10%
[2017-09-18 15:09:37.730] [I] Datastore has: 87% free space
[2017-09-18 15:09:37.733] [I] Datastore meets 10% reservation threshold.
[2017-09-18 15:09:37.735] [I] Checking that enough storage is available for the virtual disk(s)
[2017-09-18 15:09:37.737] [I] The reserved size at 10% is: 205 GB
[2017-09-18 15:09:37.738] [I] Free space available taking into account the reservation: 1557 GB
[2017-09-18 15:09:37.740] [I] The datastore will have 1556 GB of usable space remaining after the commit
[2017-09-18 15:09:37.741] [I] Datastore meets capacity requirements
[2017-09-18 15:09:37.744] [I] The Virtual Disk(s) will be located on: DATASTORE1