SC’14, the big, annual High Performance Computing (HPC) conference officially kicks off in New Orleans this evening with the opening of the exhibits. Workshops and tutorials have been held over the weekend as well: I gave a keynote presentation on Sunday at the 4th International Workshop on Network-aware Data Management (NDM’14), titled RDMA in the Cloud: Enabling high-bandwidth, low-latency communication in virtual environments for HPC (slides: NDM’14 keynote).
While I’ve been attending SC since the late 1980s and have attended each year I’ve been at VMware, this year marks the first time VMware will have an official presence on the show floor. We’ve partnered with EMC’s Office of the CTO and EMC/Isilon, and will be staffing a “Virtualization for Scientific Agility” demo station in the EMC booth.
We’ll be showing two projects of interest to the HPC community. The first is a proof-of-concept prototype created by my Office of the CTO colleague Andy Nelson that demonstrates creating and re-sizing a virtual HPC cluster in a vRealize Automation environment using Univa Grid Engine, a popular HPC batch scheduler, to illustrate the concept. Our approach leverages and extends VMware’s Hadoop provisioning system, Serengeti/BDE, to handle a more general scale-out deployment.
Our second project, in conjunction with both the EMC Office of the CTO and our partner Mellanox, demonstrates the use of InfiniBand SR-IOV to allow multiple VMs to simultaneously access a Lustre parallel file system (or other RDMA-connected file system) from a single host — an important capability if virtualization is to gain broad acceptance for HPC workloads.
Stop by the EMC booth (#2151) if you’d like to learn more about either of these demos, to talk in detail about Infiniband/RDMA performance results, or why customers are interested in building private cloud environments for running their HPC workloads.
Comments