As hybrid-cloud adoption grows, customers are increasingly looking to optimize their applications to get the most out of their VMware and AWS infrastructure. For example, in conversations with VMware cloud-solution architects, they often ask to host the vSphere Content Library (the storage container for VM images and templates) directly on Amazon S3.
The benefits of hosting the vSphere Content Library are twofold: freeing up capacity on the VMware Cloud on AWS’ high-performance local storage (low-latency NVMe SSD storage) and enhancing Content Libraries via the reliability and durability provided by AWS S3. While some customers have manually set up S3-backed vCenter Content Libraries, these configurations are not officially supported and the configuration process is quite cumbersome.
Enter the VMware Office of the CTO (OCTO) Accelerated Co-Innovation Engineering (ACE) team. As we explained in a previous post, we’re tasked with rapid execution and implementation of innovative solutions, features, and enhancements for VMware solutions. In recent weeks, the ACE team began capturing requirements to develop a new solution and user interface to enable the creation and lifecycle management of Content Libraries and their contents directly in S3, in a fashion similar to the way it’s done in vCenter.
High-level component details
The tool leverages Kubernetes Operator implementations to instantiate and manage resources alongside the open-source project Crossplane.
The key components of the solution are:
- An Operator responsible for the lifecycle management of Content Libraries
- Crossplane, which enables lifecycle management of S3 buckets and SQS queues, including access policies and linking events on bucket contents to be deposited into the queue
- An indexer service that consumes events in the queue and updates the indexes in the buckets required for vCenter support.
You might ask why you would use Operators and not a traditional configuration-management tool (CM), such as Terraform. We see a trend toward Operators, which keep constant watch and enforce instantaneous state control over managed objects, as opposed to single-run CM tools. You can read more about this trend in this blog post. But our reasoning is that if we used traditional CM tooling, we would ultimately need additional support around it to get a reconciliation loop similar to the Operator. Creating Operators is admittedly a bit more work than writing manifests or playbooks, but the outcome is more precise and faster at resolving state drift.
As shown in the diagram above, the core mechanics of the tool are open-source Operators from Crossplane, which we use to configure and manage the AWS components. We have written additional Operators to manage and configure the Crossplane custom resources, as well as for the Content Library and its index.
Through Crossplane and our own Operators, our S3 backed Content Library is not only able to safely provision infrastructure, but also to leverage the lifecycle benefits from using Kubernetes objects. The results? Customers will be able to deploy S3-hosted vSphere Content Libraries with minimal effort. The team is aiming to release the tool as a Fling in early 2021.
If you are a VMware customer with a similar request for a feature or integration, please reach out to your account team about engaging with VMware’s OCTO ACE team. We are ready to help.
Authors:
 |
|
Nicolas Vibert is Principal Cloud Solution Architect for the VMware Cloud on AWS service. |
 |
James Wirth is a Senior Member of Technical Staff at VMware within the Professional Services group. |
