VCF Remote Clusters is a new capability that was recently announced in this blog within VMware Cloud Foundation (VCF) that extends the powerful VCF cloud operating across sites to unify Edge or Remote Office Branch Office (ROBO) locations with centralized administration and control. This blog will go into details on how to utilize VCF Remote Clusters, including deployments with VeloCloud SD-WAN to ensure complete stable connectivity between the main VCF site and all remote locations.

VCF Remote Clusters extend the Cloud Foundation operational capabilities to any Edge (ROBO) and remote sites enabling these sites to operate without onsite admins. VCF operations, such as bring-up, life cycle management, adding, and removing clusters can be performed centrally from the SDDC Manager in the central or the regional data center. This provides a consistent operation and a cloud-like experience across the data center and the remote edge sites.

With VCF Remote Clusters, customers can deploy a Workload Domain or a Remote Cluster from the SDDC Manager console. This eliminates a need for having a Cloud Admin (VCF skill resource) in the remote site. Customer can also perform a full-stack life cycle management for the remote sites from the central SDDC Manager.

The latest release of VMware Cloud Foundation Edge brings capabilities that are perfect for enterprise and large retailers that are driving demand for distributed compute processing outside of the data center with use cases such as Telco 5G compute edge, IoT, machine learning robotics, and many more.

These are the requirements that resulted in the VCF Remote Clusters configuration which has just completed field validation testing in our OCTO, VMware Dell Technologies Solution (DTS) Innovation Lab within the Office of the CTO (OCTO) and is now supported with VCF 3.9 and VCF 4.0 or later releases.

Pre-requisites to Deploy and Run VCF Remote Clusters

  • A reliable and stable WAN link with minimum 10Mbps bandwidth with less than 50ms of end-to-end latency*.(Note: It is recommended to have primary and secondary WAN links between central VCF site to VCF Remote Clusters site.
  • Edge site network should be reachable to VCF Management site to ensure connectivity of management components such as vCenter, SDDC Manager, NSX-T Manager etc.
  • DNS and NTP Server is available locally or they are reachable to Edge site from Central site.
  • A DHCP server must be available for  the NSX-T host overlay (Host TEP) VLAN of the workload domain. When NSX-T creates Edge Tunnel End Points (TEPs) for the VI workload domain, they are assigned IP addresses from the DHCP server. DHCP Server should be available locally at the Edge site.

Solution Design

The following figure shows the deployment architecture of the VCF Remote Clusters across two physically separated sites: Management Data Center and Edge using VeloCloud SD-WAN. Note that VeloCloud is not required for VCF Remote Clusters to be operational however, VeloCloud SD-WAN provides enhancements to a VCF Remote Cluster deployment that is complementary to the native networking provided by NSX.  VeloCloud delivers increased network agility and cost reduction by using a software-defined approach to abstracted network hardware and by transporting characteristics from the applications that are using the VCF Remote Cluster network.

Figure:1 VMware Cloud Foundation Edge Architecture

  • In this design, we have VeloCloud SD-WAN virtual edge in both sites (Management Data Center and Edge) and they are connected over internet. Two site connectivity bandwidth is 10Mbps and latency is 34ms.
  • These two sites are physically separated: one being hosted at Washington State and another at Santa Clara, California in our OCTO DTS Innovation Lab.
  • VeloCloud SD-WAN provides the L3 IPsec tunnel between two sites. Network subnets of both the sites are routable and they are reachable to each site except VSAN and vMotion.
  • Both the site having two ToR switches and they are connected to VeloCloud SD-WAN for north bound connectivity. And all the compute nodes are connected to each site-specific ToR switches.
  • Common services like, Active Directory, DNS and NTP are hosted in Management Data Center site and they are reachable to Edge site.
  • VMware Cloud Foundation with a Management Workload Domain is deployed on 4 node Dell PowerEdge R730 in Mgmt. Data Center. Mgmt. Data Center also hosts all management components such as SDDC Manager, VMware vCenter, NSX Manager, Active Directory, DNS and NTP server.
  • VMware Cloud Foundation SDDC Manager is used to deploy a remote Edge Workload Domain with NSX-T on 3 node Dell PowerEdge R640 in Edge site.
  • Edge WLD management components such as vCenter and NSX-T Manager is hosted on Mgmt. Data Center.
  • Edge WLD – NSX-T Edges are deployed in Edge site for north-south connectivity.
  • VCF operations like lifecycle management, adding cluster, and removing cluster is managed centrally from Management Data Center.

Network Design

Networking is a key component for VCF Remote Cluster deployments to be successful and proper network planning is fundamental to success of any network edge deployment.  VCF Remote Cluster requires stable, consistent connectivity to ensure seamless operations and administration. Figure 2 below shows how we designed and implemented the network for these two sites (Management Data Center and Edge) architecture in our OCTO DTS Innovation Lab.

Figure 2: Network Architecture

Kubernetes at the Edge

vSphere 7 with Kubernetes is now available as part of the VMware Cloud Foundation 4 release and customers can extend their Kubernetes operating model to Edge with VCF Remote Clusters. The deployment of vSphere with Kubernetes is no different and the customer experience is similar to deploying vSphere with Kubernetes in the central data center. For additional information, review the VCF Operations and Administration Guide.

External Storage at the Edge

There are many use cases where data locality is needed, such as video surveillance, AI/ML data analytics and IoT where customers need external storage at the Edge location. In this situation, VMware Cloud Foundation Remote Cluster does support IP/FC based external storage as primary and secondary storage at the Edge site. Figure 3 below shows the architecture of how a customer can connect a Dell EMC Unity network attach storage in a remote site with VMware Cloud Foundation Remote Cluster.

Figure 3: External Storage with VMware Cloud Foundation Edge

Figure 3: External Storage with VMware Cloud Foundation Edge

Below is the technical specification of two site VCF Remote Cluster validation.

Technical Specification:

Site-Mgmt. Data Center:

Site-Edge:

 

Wrapping Up

VCF remote cluster is a new Cloud Foundation feature that extends VCF operational capabilities to remote edge sites in order to provide a consistent experience, delivering full-stack processing power as close to the data source as possible.  The testing performed by the Office of the CTO team showed that the VCF Edge deployment provides a robust, scalable system that distributes compute processing beyond the data center to remote edge locations. Stay tuned to the VMware Cloud Foundation website for additional updates to this exciting new capability.

Additional Resources:

What’s New with VMware Cloud Foundation 4.1

Extending the Power of VMware Cloud Foundation to the Edge