Data is at the core of everything that VMware customers do. Data is the lifeblood of the modern enterprise, and their success is increasingly determined by the ability to extract value from it. This means that the value of cloud infrastructure(s) is expanding beyond just storing data into managing, sharing, and analyzing it.
Today, the reality is that our customers must cope with several types of infrastructure, often resulting in silos. That siloing occurs not only in physical infrastructure but also in operational models, security, and compliance. Running one’s applications across such silos is complex, expensive, and risky. We call it “Cloud Chaos.” VMware’s strategy is to offer a layer of common abstractions for operations and a common platform for running any application on every type of cloud to become “Cloud Smart.”
One aspect of those common abstractions has to do with data, which is at the epicenter of our customers’ digital transformation.
VMware already offers a variety of multi-cloud options for storage, storage management, and data protection. However, the way data is consumed and managed is changing drastically. Applications (and developers) consume data using higher-level abstractions – for example, SQL APIs as opposed to POSIX file systems. Many modern applications do not implement a data persistence layer. Instead, they assume it is offered as part of the cloud services they depend on. In fact, the industry (VCs, analysts, customers) considers data services, such as databases, data lakes, and data warehouses, to be part of the “Cloud infrastructure.”
In this article, we outline VMware’s strategy for multi-cloud data management. First, we cover the new extensions to VMware’s main cloud platform (vSphere) that aim at facilitating core data services on VMware clouds, starting with database services. Then, we discuss the longer-term opportunity for interconnecting such data services and offer more comprehensive data management solutions through a combination of in-house R&D and a partner ecosystem.
Core Data Services
Today, VMware customers manage an increasing number of database types and instances. This trend is fueled by the digital modernization of enterprises and the emergence of modern applications rapidly becoming central to their businesses. The customer requirements start with relational databases but also include or expand to in-memory databases, data warehouses, key-value stores, streaming and messaging systems, etc. Collectively, we call them “core data services.”
VMware already has a comprehensive portfolio of data software assets from Pivotal, including VMware Greenplum, SQL, GemFire, and RabbitMQ. Customers use those products to store, move, process, and query data resources in real-time and at a massive scale, on-premises, on sovereign clouds, and on public clouds.
More recently, VMware expanded its portfolio with software aimed at solving a pressing problem for our customers – how to manage a proliferating database estate, while supporting the agility and cloud-native consumption model required by application developers. Indeed, a typical enterprise IT organization has gone from managing a handful of Oracle DBs (databases) 10 years ago to juggling sprawling DB farms with hundreds of different types of DBs today. At the same time, application teams want to provision databases programmatically and consume them with certain service level objectives (SLOs) ensured by IT. Also, database management is increasingly performed primarily by generalist IT professionals under the guidance of a diminishing number of DBAs (database administrators) who struggle to scale.
VMware is shipping the first version of a product that helps our customers tackle the two problems above. VMware Data Services Manager (DSM) is a native extension of vSphere and is integrated with other VMware management products. Customers can use it to manage three of the most popular DBs: Postgres, MySQL, and SQL Server (*). DSM features serve three primary personas as shown below:
- The DBA sets up database templates according to expected SLOs (service level objectives) and establishes policies for data protection and availability for different parts of the lifecycle of a database. The more tedious tasks of database lifecycle are automated and coordinated with infrastructure management.
- The IT admin stays in control of infrastructure and resource management. She has end-to-end visibility of key infrastructure and database metrics, and with that, she can effectively deliver the SLOs required by database consumers.
- The app developer can create apps at scale and speed with programmatic self-service capabilities. He can integrate database provisioning and even relevant management features (e.g., replica creation) with their development processes and CI/CD pipelines. He depends on the IT team to deliver the desired SLOs and to comply with the various data protection and governance requirements.
In a nutshell, the goal of the DSM is to empower the IT/DBA teams to deliver a true DBaaS (database as a solution) offering on-premises to their internal customers, who are typically the application teams in the lines of business.
Changes in the macroeconomic environment motivate customers to be much more conscientious of their cloud spend, bringing VMware into focus as a cost-effective on-premises alternative for data services. With increasingly cost-conscious customers, we also see a skill consolidation in IT teams, wherein day-to-day DB operations increasingly fall on the shoulders of IT generalists, whereas the traditional DBA role becomes more focused on database architecture and on how to extract value from data for internal consumers.
Customer Use Case:
- Problem: a tier-1 healthcare services provider in Europe was facing severe “data sprawl”: Diversifying developer and service requirements resulted in proliferating DB farms on vSphere that outgrew their DBA resources, while their attempt at a custom automation solution failed. This is an increasingly common customer pain pattern.
- Solution: with VMware Data Services Manager, this customer is transitioning from their homegrown tooling to provide coveted self-service capabilities for developers and necessary automation and monitoring for IT.
Our vision is to help customers to build, deploy and run their data infrastructure across any cloud, whether private or public, with the same operational model and SLOs for performance, business continuity, and security. We are essentially making data as easy to consume and manage as we did for compute, storage, and networking in the past.
At this point, DSM delivers a software product that customers use to offer DBaaS to their internal “end” customers (typically, as we said, the application teams). We are also exploring other operational models, For example, VMware-managed data services. We are trialing such data services internally before we decide if and how we offer them to customers. We demonstrated some of those ideas with a tech preview of Project Moneta during a 2022 VMware Explore Europe session, “Data Infrastructure for Modern Applications,” and are looking forward to meeting you at VMware Explore 2023 to share our progress.
What’s Next?
Our long-term vision is to make VMware the de facto multi-cloud data management leader, empowering our customers to fully extract value from their data without being constrained by any specific cloud or vendor-vertical solution. VMware can help our customers achieve this goal by building a data management platform that enables customers to locate, access, and utilize data across different clouds and data repositories, from various operational databases to data warehouses and data lakes. This is the third wave in the picture of Figure 4 below:
Multi-cloud data management goes beyond the core data services (databases) that our platform supports today. It includes technologies and solutions for data cataloging and discovery, lineage tracking, metadata management, cross-cloud data access and transfer, governance, and compliance, etc.
This is a complex problem to solve: emerging requirements include reading data from multiple sources, even across clouds, and optionally writing it to multiple destinations. And do that while supporting different generations of query engines, data formats, and schemas; manage resource contention and rate limiting across the data sources; optimize for cloud economics, including the high egress charges of public clouds (and support more customer-friendly initiatives); assure data recency and data quality metrics. In addition, writing data adds challenges like solving data routing, transaction management, and business logic transformation.
Our goal is to evolve our data platform in a way that is leveraged by a broad partner ecosystem bringing increasing value to customers in addition to the solutions and services built internally at VMware.
If you are interested in learning more about VMware’s Data Strategy, contact your account team to schedule a follow-up discussion with the authors.
Comments