Deploying HCX (VMware Hybrid Cloud Extensions) is considered to be complex and difficult by most. It doesn’t help that it’s usually one of those things you’d only do once so it doesn’t pay to spend a lot of effort to learn. However, as with everything it’s not hard once you know how to do it. This video aims to show how to deploy HCX both in VMC (VMware Cloud on AWS) and in the on-premises DC or lab.
It uses both the method of creating the service mesh over the internet as well as how to create it over a private connection, like DX (AWS Direct Connect) or a VPN.
A VPN cannot be used for L2 Extension if it is terminated on the VMC SDDC. In this tutorial I’ll use a VPN which is terminated on an AWS TGW which is in turn peered with a VTGW connected to the SDDC we’re attaching to.
I printed in white PLA using the 0.2mm quality preset on a Prusa i3 MK3s
Trying out the fit of a few silver cherry compatible switches after having painted purple using Tamiya TS-24 model paint
Soldering the diodes
For the wiring I used the diagram by Nick Green shown here.
For each row I use the diodes own wires as connectors between the keys. Solder the brown side of the diode to the key and use the black-side wire to hook up to the next key.
Soldering the vertical connectors
To connect the keys on the vertical side I use AWG22 copper wire with different colors to keep them separate more easily. AWG24 might have been better but this is what I had available at home.
I start by laying out the wire over the keys and then using a permanent marker to mark where they should have the insulation removed
Then I use a wire stripper to remove the insulation where the wire is marked. That way we can connect the same wire to multiple keys without having to cut the wire. The exposed part of the wire can also be pushed down over the key pin to get it to stay put while being soldered into place
Soldering largely done! Having some helping “hands” is highly recommended
Soldering pins to the Arduino Pro Micro
To make soldering of the pins easier I simply push them into a breadboard for support
Attaching the wires to the Arduino
To avoid the hassle of soldering each Arduino pin to the keyboard wires, and also to make it easy to replace the Arduino / wires if required, I use a crimping tool and some XH connectors.
Once the wires are attached to the XH connectors they can easily be connected to the pins of the Arduino. Some velcro keeps everything nice and tidy.
Adding the 3.5mm audio jacks
I solder VCC and GND wires to the black 3.5mm audio module and attach them to the corresponding pins on the Arduino using another XH connector. The data pin attaches to D3.
The two halves can now be connected using the 3.5mm audio cable (gray in the picture)
Quick (?) steps for connecting a Mikrotik router in an on-premises lab or DC to an AWS VPC using a VPN. All commands done over AWS CLI and Mikrotik CLI.
Note: The values for tunnel IP addresses and secrets etc. can be found in your VPN configuration file (downloaded later). Please don’t use the ones in this guide or an IT fairy will jump to her death from a VAX system in some remote DC. The values used here are already invalid as the resources have been deleted by the time of writing. Do think of the fairies though.
In this case the Mikrotik is not directly attached to the internet. It goes via an ISP router. If your setup is the same, please configure port forwarding for ESP, UDP port 500 and UDP port 4500 from the ISP public interface to the Mikrotik router as per the diagram.
If the Mikrotik is directly attached to the internet please open the firewall ports accordingly for ESP and UDP 500 / 4500.
Creating the VGW (Virtual Private Gateway but called vpn-gateway on the CLI). I used 65011 here for the AWS-side ASN but feel free to use something different as long as it is supported
Download the router configuration from the AWS console. Navigate to VPC and select Site-to-site VPN connection on the left-hand list. Pick the connection we just created and download the config as a text file
That’s it. The AWS side is done for now. We’ll need to add return routes from the VPC to the on-prem networks later but for now we can continue on to the Mikrotik configuration
Open the downloaded router configuration text file and SSH to the Mikrotik router. I use RouterOS 6.49.6 for this guide (latest at time of writing). An AWS VPN uses two tunnels. We have to configure both but will disable one of them later. Mikrotik doesn’t support dual active tunnels to AWS.
Create the IP addresses for the VPN tunnels. Search from the top of the file and look for “Customer gateway Inside Address”. The first 169.254.x.x IP will be for Tunnel 0. A second IP will be listed further down for Tunnel 1. We use a /30 subnet mask for the tunnel IPs.
Use your router outside interface. Mine is “sfp-sfpplus1” for this example
[admin@MikroTik] > ip address add address=169.254.88.206/30 interface=sfp-sfpplus1
[admin@MikroTik] > ip address add address=169.254.19.154/30 interface=sfp-sfpplus1
When migrating from an on-premises DC to VMware Cloud on AWS it is usually recommended to use Hybrid Cloud Extension (HCX) from VMware. However, in some cases the IT team managing the on-prem DC is already using Veeam for backup and want to use their solution also for the migration.
They may also prefer Veeam over HCX as HCX often requires professional services assistance for setup and migration planning. In addition, since HCX is primarily a tool for migrations, the customer is unlikely to have had experience setting it up in the past and while it is an excellent tool there is a learning curve to get started.
Migrating with Veeam vs. Migrating with HCX
Veeam Backup & Recovery
VMware Hybrid Cloud Extension (HCX)
Licensed (non-free) solution
Free with VMware Cloud on AWS
Arguably easy to set up and configure
Arguably challenging to set up and configure
Can do offline migrations of VMs, single or in bulk
Can do online migrations (no downtime), offline migrations, bulk migrations and online migrations in bulk (RAV), etc.
Can not do L2 extension
Can do L2 extension of VLANs if they are connected to a vDS
Can be used for backup of VMs after they have been migrated
Is primarily used for migration. Does not have backup functionality
Support for migrating from older on-prem vSphere environments
At time of writing, full support for on-prem vSphere 6.5 or newer. Limited support for vSphere 6.0 up to March 12th 2023
What we are building
This guide covers installing and configuring a single Veeam Backup and Recovery installation in the on-prem VMware environment and linking it to both vCenter on-prem as well as in VMware Cloud on AWS. Finally we do an offline migration of a VM to the cloud to prove it that it works.
The guide assumes the following is already set up and available
On-premises vSphere environment with admin access (7.0 used in this example)
Note: I initially used 2 vCPU, 4GB RAM and 60 GB HDD for my Veeam VM but during the first migration the entire thing stalled and wouldn’t finish. After changing to 4 vCPU, 32Gb RAM and 170 GB HDD the migration finished quickly and with no errors. Recommend to assign as much resources as is practical to the Veeam VM to facilitate and speed up the migration
One VMware Cloud on AWS (VMC) Software Defined Datacenter (SDDC)
Private IP connectivity to the VMC SDDC
Use Direct Connect (DX) or VPN but it must be private IP connectivity or it won’t work
For this setup I used a VPN to a TGW, then a peering to a VMware Transit Connect (VTGW) which had an attachment to the SDDC, but any private connectivity setup will be OK
A test VM to use for migration
Downloading and installing Veeam
Unless you already have a licensed copy, sign up for a trial license and then download Veeam Backup and Recovery from here. Version 22.214.171.1246 used in this guide.
In your on-premises vSphere environment, create or select a Windows Server VM to use for the Veeam installation. The VM spec used for this install are as follows:
Run the install with default settings (next, next, next, etc.)
Register the on-prem vCenter in Veeam
Navigate to “Inventory” at the bottom left, then “Virtual Infrastructure” and click “Add Server” to register the on-prem vCenter server
Listing VMs in the on-prem vSphere environment after the vCenter server has been registered in the Veeam Backup & Recovery console
Switching on-prem connectivity to VMware Cloud on AWS SDDC to use private IP addresses
For this setup there is a VPN from the on-premises DC to the SDDC (via a TGW and VTGW in this case) but the SDDC FQDN is still configured to return the public IP address. Let’s verify by pinging the FQDN
Switching the SDDC to return the private IP is easy. In the VMware Cloud on AWS web console, navigate to “Settings” and flip the IP to return from public to private
Ping the vCenter FQDN again to verify that private IP is returned by DNS and that we can ping it successfully over the VPN
All looks good. The private IP is returned. Time to register the VMware Cloud on AWS vCenter instance in the Veeam console
Registering the VMC vCenter instance with Veeam
Just use the same method as used when adding the on-premises vCenter server: Navigate to “Inventory” at the bottom left, then “Virtual Infrastructure” and click “Add Server” to register the on-prem vCenter server with Veeam
After adding the VMware Cloud on AWS SDDC vCenter the resource pools will be visible in the Veeam console
Now both vSphere environments are registered. Time to migrate a VM to the cloud!
Migrating a VM to VMware Cloud on AWS
Below is both a video and a series of screenshots describing the migration / replication job creation for the VM.
Creating some test files on the source VM to be migrated
Navigate to “Inventory” using the bottom left menu, click the on-premises vCenter server / Cluster and locate a VM to migrate in the on-premises DC VM inventory. Right-click the VM to migrate and create a replication job
When selecting the target for the replication, be sure to expand the VMware cloud on AWS cluster and select one of the ESXi servers. Selecting the cluster is not enough to list up the required resources, like storage volumes
Since VMC is a managed environment we want to select the customer-side of the storage, folder and resource pools
If you checked the box for remapping the network is even possible to select a target VLAN for the VM to be connected to on the cloud side!
Select to start the “Run the job when I click finish” and move to the “Home” tab to view the “Running jobs”
The migration of the test VM finished in less than 9 minutes
In the vCenter client for VMware Cloud on AWS we can verify that the replicated VM is present
After logging in and listing the files we can verify that the VM is not only working but also have the test files present in the home directory
Thank you for reading! Hopefully this has provided an easy-to-understand summary of the steps required for a successful migration / replication of VMs to VMC using Veeam
With CentOS being less than attractive to use now when Red Hat has changed how it is updated, the Amazon AMI2 Linux distribution can be an excellent alternative.
However, when deploying an Amazon AMI2 on vSphere for the first time there are a few hoops to jump through. This video shows how to create a golden image and deploy it with Terraform in less than 15 minutes