The topic of L2 extensions with HCX comes up often when architecting HCX on AVS. Having the ability to do an L2 extension from on-premises to AVS allows customers to migrate VMware based workloads without re-IP the VM while at the same time maintaining default gateway and local security policies throughout the migration process.
In this blog post, we will walk you through the process of setting up an L2 extension with HCX and how to terminate and migrate your gateway from on-premise to the cloud once the migration is over.
The image below illustrates how our on-premise HCX appliance uses our express route connection to establish a tunnel with our HCX instance on AVS.
The green network represents our on-premises management network. The management components on AVS, including NSX, HCX, and vCenter are automatically created during the AVS provisioning process.
Our focus for most of this blog will be on the blue network representing our on-premises L2 segments for our compute workloads.
Note: For simplicity purposes, we decided only to represent a single L2 segment for our compute workloads; typically, customers would have multiple; however, the process for stretching and un-stretching L2 networks is the same regardless of the number of networks.
Note: HCX does not currently support extending your on-premises networks used for vSphere management, such as vMotion and HCX components. For more details, please read the HCX user guide. https://docs.vmware.com/en/VMware-HCX/services/user-guide/GUID-BFD7E194-CFE5-4259-B74B-991B26A51758.html
In this blog post, we will not be covering the installation process for HCX. If you want more information on this topic, please see https://docs.microsoft.com/en-us/azure/azure-vmware/tutorial-deploy-vmware-hcx
From your on-premises vCenter, log into your HCX instance.
From the HCX UI, click on “Network Extension.”
Click on “Extend Networks”
Select the network you want to extend. In this case, we will be stretching our Contoso port group.
Note: This network can be VLAN backed, or VXLAN backed. The on-premise requirements for stretching networks with HCX are VDS (Virtual Distributed Switch), vSphere 5.5 or higher, express route, and an Intel-based processor.
Next, we need to enter the IP address of the gateway we want to extend into AVS. This process is the same regardless if your gateway is on a physical device or a virtual instance such as NSX. In our environment, the default gateway for our Contoso network is 10.255.255.1/24.
I will be entering that information into the “Gateway IP Address” field and selecting my NSX-T1 router as the destination first-hop router in AVS. Keep in mind that this will not create another gateway on AVS; your default gateway will remain on-premise during the stretch.
Note: For the “Extension Appliance” field, you more than likely will leave the default settings; however, if you want to stretch multiple portgroups and the aggregated bandwidth exceeds 2.5gbps, then it will be a good idea to deploy additional extension appliances to distribute the load.
The extension process should take a few minutes to complete.
At this point, our network is stretched into AVS. Please see the below diagram illustrating our network stretch.
As you can see, our default gateway is still on-premises. Keeping the gateway on-premises gives us the ability to continue to use our on-premises security stack while at the same time reducing significant network changes during the migration process.
While HCX provides the feature of having an L2 stretch, it is essential to note that the desired outcome is to eventually decommission the tunnel and the L2 extension once the migration is complete.
As long as the default gateway remains on-premises, any type of routing will have to be hair-pinned back to our local gateway; this will cause slower network speeds and egress charges through the express route.
As you can see from the diagram below, our red network is AVS native (non-stretched). If a VM on our blue network wants to communicate to a VM on the red network, the traffic will have to transverse through on-premises.
Once you have finished migrating every VM out of your stretched portgroup, you want to decommission the L2 stretch and migrate your gateway into AVS; this will eliminate the hair-pinning of the traffic.
Note: Make sure you have pre-populated your security policies required to protect your workloads in your NSX AVS environment.
The image below illustrates our desired network layout.
To migrate our default gateway, we need to “unexted” our stretched network from our on-premise HCX environment.
Select “Connect cloud network to cloud edge gateway after unextending” this will create a gateway in our AVS environment and attach it to our T1 router.
Once the network is unstretched, you should see the network being advertised back to your on-premise environment through BGP via your express route connection.
Note: This process should take a few minutes to complete.
Note: This process should cause a network outage since you need to decommission the gateway from on-premises to eliminate multiple routes to the same network. It is also recommended that you stop advertising the on-prem decommissioned network to reduce the downtime that BGP reconvergence would take. Depending on your setup you might also need to edit your roue filters to allow the new advertised subnet from AVS after unextending.
