Tuning tips to run low-latency applications in virtual machines

First off, a little bit about myself. I’m Bhavesh Davda, presently a member of the Office of the CTO here at VMware, looking at areas like low latency applications, high rate packet processing, high performance computing, SR-IOV and RDMA, which have historically been challenging to virtualize. Until recently, I managed the vSphere Networking R&D team within VMware, where my team worked on the entire virtual networking stack from guest OS drivers for paravirtual NICs, to virtualization of paravirtual and emulated NICs in the hypervisor, to efficiently implementing I/O down to the physical NIC drivers and our own TCP/IP stack in the vmkernel hypervisor for uses like VMotion, Fault-Tolerance and NFS at the hypervisor level.

For the longest time, it was unusual to hear the words “low latency applications” and “virtualization” used in the same sentence.

That’s because historically some low latency workloads have such challenging expectations on the latency front (sub-microseconds to a few tens of microseconds), that they have to be heavily tuned, at all levels of the application stack from the interconnect fabric, to I/O devices, to BIOS settings on servers, to the operating system tunables, to using programming techniques like OS bypass with RDMA/VERBs or custom system calls, to achieve the lowest possible latencies.

And historically, while virtualization has been considered a wonderful tool for consolidating your typical enterprise-class application workloads (See “IDC White paper – Virtualising Tier 1 Applications“), it hasn’t been looked at seriously for running latency-sensitive applications in virtual machines due to perceived and/or real virtualization overheads.

However, in the last few months, as can be seen from Josh’s excellent blog post: Mendel says: It is time for low latency, where he references the HotOS ’11 paper from Stanford, and from the traction low-latency interconnects like Infiniband and 10GbE are gaining in verticals like the financial sector and HPC space, there has been a lot of excitement and demand for running such lowlatency applications in virtual machines.

Other posts by

IEEE Hot Interconnects Invited Talk: NFV in Cloud Infrastructure – Ready or Not

IEEE Hot Interconnects is the premier international forum for researchers and developers of state-of-the-art hardware and software architectures and implementations for interconnection networks of all scales, ranging from multi-core on-chip interconnects to those within systems, clusters, data centers and clouds. This year, Hot Interconnects was held in Santa Clara, CA from August 26-28. I’m grateful […]

Tuning ESXi for NFV Workloads

Network Functions Virtualization (NFV) enables the efficiency and agility benefits that come from untethering network functions from proprietary appliances and moving them to a virtualized infrastructure running over hypervisors and COTS hardware. Enterprise IT departments have been realizing such benefits for a number of years but the telecommunications environment brings new workloads with different requirements. The vSphere ESXi hypervisor provides […]

Low Latency on vSphere at VMworld 2012

I was excited to have one of our partners, Mellanox, extend a special invitation for me to speak at their Expo booth during VMworld 2012 in San Francisco. They had allocated a 30 minute slot for a Theater Presentation and promoted my presentation via nice giveaways and postcards distributed in nearby VMworld hotels. I used […]