The speed, scalability, and seamless experience that users and organizations have come to expect from modern apps create a complex challenge for service providers — one that is increasingly answered by the collaborative efforts of an organized, actively managed open-source community. This post highlights examples of this model in action and explores the best practices gleaned during their inception and development.
The VMware Innovation Network (VIN) was conceived in China in 2018 and shares the innovative spirit of earlier projects, such as the 10-year-old Cloud Foundry from VMware (then Pivotal), and the seven-year-old Project Harbor, which originated from VMware’s China R&D. Harbor was open-sourced in 2016 and was the first open-source project donated by VMware to the Cloud Native Computing Foundation (CNCF). It was also the first project from China’s open-source community. CNCF itself has blossomed into the largest and most popular sub-foundation within the Linux Foundation (LF). The China cloud-native community is now also one of the largest cloud-native developer communities in the world.
Naturally borderless open-source models let us tap into resources to deliver solutions in ways that previously weren’t possible. This has also expanded the origins of innovation far beyond Silicon Valley.
Open-source innovation takes a new path
In the venture-capital world, every product needs to fit a market, or vice versa, for a startup to be successful. The open-source innovation-and-development model also needs a prospective “market” — a user who will consume what the project produces.
Open-source development often emerges from simple ideas and primitive projects with early adopters/users in the community. At first, these projects may not appear to have a commercial product or a viable market. However, this project-user fit is a starting point that can lead to a product-market fit and eventually to customers and profit.
VIN is an open-innovation network that includes both co-innovators and co-enablers. Co-innovators may include customers and partners who want to collaborate on projects, startups bringing new ideas to market, and academic institutions. There are currently more than 20 customer/partners, more than 60 startups, and 10 academic institutions acting as co-innovators in this network. Co-enablers are partners that help accelerate and enable our co-innovators, such as the Linux Foundation (LF), Apache, city or regional incubators and accelerators, corporate and financial venture capitalists, and private-equity firms.
Following are some specific case studies that involve VMware’s experience of open-source collaboration in action.
FATE and WeBank
The large-scale Federated Machine Learning (FML) open-source project, Federated AI Technology Enabler (FATE), demonstrates the value of the partnership model. China’s WeBank — one of the world’s largest digital banks — required machine learning that protected data privacy but leveraged both “horizontal” (size of data sample) and “vertical” (number of variants and depth of analysis) ML approaches. VMware co-innovated with WeBank, enabling FATE to run in a cloud-native environment via Project KubeFATE, so it could operate in the cloud.
FATE trains machine-learning models with data from multiple sources without compromising regulation of data-privacy protection. This means it has great potential for use cases across industries beyond banking and finance.
The co-innovator model also drives value back to the innovation community. WeBank donated FATE to LF in 2019. VMware has since become the second-biggest contributor to the LF project, after WeBank.Tencent Cloud is the third-biggest and there are many more contributors from the FML community around the world. Whereas open-source software used to be thought of as a “free lunch” (at least from a user perspective), things have changed rapidly over the past decade. Now it has contributors from startups, large enterprises, and grass-roots developers. For example, Clustar, a VIN startup and FATE partner, is working with WeBank and VMware to launch a FATE-based FML solution on VMware Cloud Foundation with Tanzu.
Recent cloud-native and privacy-computing events have featured 10,000+ online and offline attendees from the China cloud-native and FML communities.
TiDB and PingCAP
TIDB, a cloud-native New SQL company, unicorn startup, and VIN member, delivers a distributed hybrid transaction analytic processing (HTAP) database to the market. In 2018, Project TiKV (the storage layer of TiDB) became the second Chinese open-source project contributed to CNCF in 2018. PingCap (who started developing TiDB soon after its founding) and VMware have been co-leading to evangelize cloud-native application and data technology. They have since been joined by other VIN partners, including big names, such as AliCloud, Tencent Cloud, JD.com, NetEase, and startups such as Alauda and CloudChef.
Acting as co-innovators, a small group of people from PingCap and VMware field sales and R&D jointly created a solution based on HTAP for modern apps based on TiDB and Tanzu. The partnership showed that open-source and open collaboration amongst organizations of all sizes and locations can create business value to address customer pain points.
EdgeX Foundry, Intel, and EMQ X Kuiper
More than 50 companies created the project EdgeX Foundry. Its goal is to simplify and standardize industrial IoT edge computing by bringing stakeholders together, while allowing each vendor to create its own products based on the common codebase.
Project EdgeX Foundry emerged to solve a challenge across industrial, enterprise, and consumer IoT: the lack of a standards-based platform. IoT companies are duplicating efforts by writing their own code and developing their own protocols. The result is fragmentation and interoperability challenges.
VMware is one of the key members of EdgeX Foundry. In 2019, VMware co-founded the EdgeX Foundry China Project with Intel. Now it has key contributors and users from Alibaba, Baidu, Tencent, ARM, Thundersoft, EMQ, Jiangxing Intelligence, and Advantec. Recent community events have counted 10,000+ attendees online and offline. Many VIN member partners have launched IoT/Edge solutions based on EdgeX Foundry.
Project EMQ X Kuiper, incubated and accelerated by the EdgeX Foundry China project, was donated by EMQ into EdgeX as the new EdgeX reference implementation rules engine (edge analytics). VIN startup member EMQ, a MQTT message broker and stream-processing database provider, has become a great user-turned-contributor, giving to and benefiting from the community at the same time.
FlowGate and Quarkdata
Quarkdata is a Beijing-based AIoT/edge startup and VIN member. It incorporated the open-source project FlowGate, originated by VMware China, to engineer facility and IT data convergence to deliver more efficient control of air conditioning and datacenter power needs. By connecting and analyzing the building infrastructure and datacenter sensors to the computing infrastructure, the solution has achieved significant energy savings and carbon reduction. For example, a Telco IDC in Shanxi Province has cut power consumption for cooling by 30%.
According to a study by Greenpeace East Asia and the North China Electric Power University, power consumption of China’s data centers is forecasted to increase 66% over the five years to 2023. The savings conferred by Quarkdata’s DeepCooling project will translate into significant business benefit to the enterprise, as well as positive impact to the environment.
VMware acted as both a co-innovator and co-enabler with Quarkdata. The three-way collaboration (with Intel as a third partner) was key to bringing the DeepCooling solution to market.
Looking to the future
Well-organized open-source collaboration is key to the type of innovation necessary to tackle the complexity (such as FATE’s federated datasets and privacy requirements and EdgeX Foundry’s distributed IoT/edge networks) inherent in modern application and data requirements. The VIN model allows projects to tap the leading expertise available in the global innovation community to work towards the common good. We hope to share best practices from VIN with the global community to facilitate collaboration and open co-innovation in areas such as cloud-native modern apps and edge computing for AIoT.