Tuning tips to run low-latency applications in virtual machines

First off, a little bit about myself. I’m Bhavesh Davda, presently a member of the Office of the CTO here at VMware, looking at areas like low latency applications, high rate packet processing, high performance computing, SR-IOV and RDMA, which have historically been challenging to virtualize. Until recently, I managed the vSphere Networking R&D team within VMware, where my team worked on the entire virtual networking stack from guest OS drivers for paravirtual NICs, to virtualization of paravirtual and emulated NICs in the hypervisor, to efficiently implementing I/O down to the physical NIC drivers and our own TCP/IP stack in the vmkernel hypervisor for uses like VMotion, Fault-Tolerance and NFS at the hypervisor level.

For the longest time, it was unusual to hear the words “low latency applications” and “virtualization” used in the same sentence.

That’s because historically some low latency workloads have such challenging expectations on the latency front (sub-microseconds to a few tens of microseconds), that they have to be heavily tuned, at all levels of the application stack from the interconnect fabric, to I/O devices, to BIOS settings on servers, to the operating system tunables, to using programming techniques like OS bypass with RDMA/VERBs or custom system calls, to achieve the lowest possible latencies.

And historically, while virtualization has been considered a wonderful tool for consolidating your typical enterprise-class application workloads (See “IDC White paper – Virtualising Tier 1 Applications“), it hasn’t been looked at seriously for running latency-sensitive applications in virtual machines due to perceived and/or real virtualization overheads.

However, in the last few months, as can be seen from Josh’s excellent blog post: Mendel says: It is time for low latency, where he references the HotOS ’11 paper from Stanford, and from the traction low-latency interconnects like Infiniband and 10GbE are gaining in verticals like the financial sector and HPC space, there has been a lot of excitement and demand for running such lowlatency applications in virtual machines.