Performance of RDMA and HPC Applications in VMs using FDR InfiniBand on VMware vSphere

Bare-metal and virtual MPI latencies for a range of ESX versions

Bare-metal and virtual MPI latencies for a range of ESX versions

Customers often ask whether InfiniBand (IB) can be used with vSphere. The short answer is, yes. VM Direct Path I/O (passthrough) allows InfiniBand cards to be made directly visible within a virtual machine so the guest operating system can directly access the device. With this approach, no ESX-specific driver is required — just the hardware vendor’s standard native device driver for Linux or Windows. VMware supports the VM Direct Path I/O mechanism as a basic platform feature and our hardware partners support their hardware and device drivers.

Those interested in InfiniBand are, of course, primarily concerned with performance — can we achieve performance competitive with bare metal? We’ve recently published a technical white paper that answers this question for two constituencies — those interested in consuming the low-level RDMA verbs interface directly (typically, distributed databases, file systems, etc.) and those interested in running HPC applications, which typically run on top of an MPI library which in turn uses RDMA to control the InfiniBand hardware.

In addition to providing detailed RDMA and MPI micro-benchmark results across several ESX versions, the paper includes several examples of real HPC application performance to demonstrate what can be achieved when running MPI applications on vSphere.

The paper, titled Performance of RDMA and HPC Applications in Virtual Machines using FDR InfiniBand on VMware vSphere, is available here.

Other posts by

vSphere Scale-Out for HPC and Big Data

I’m very excited that we’ve announced vSphere Scale-Out this week at VMworld here in Las Vegas. This new vSphere edition is specifically and exclusively designed for running HPC and Big Data workloads. This is an important development in our work to offer compelling virtualization solutions for these two emerging workload classes. Our strategy for addressing […]

Three Extreme Performance Talks from the Office of the CTO at VMworld USA

The Office of the CTO will be presenting three talks in the unofficial “Extreme Performance” series at the upcoming VMworld 2017 conference in Las Vegas. In addition, one of these talks will be delivered at VMworld Europe in Barcelona. Each of these talks focuses on important aspects of pushing the envelope to achieve high performance […]

How to Enable Compute Accelerators on vSphere 6.5 for Machine Learning and Other HPC Workloads

As our CTO Ray O’Farrell recently mentioned, VMware is committed to helping customers build intelligent infrastructure, which includes the ability to take advantage of Machine Learning within their private and hybrid cloud environments. As part of delivering this vision, the Office of the CTO collaborates with customers and with VMware R&D teams to ensure the […]