Hadoop Performance on vSphere 5.1

Hadoop logoWe’ve just published a third Hadoop performance paper, written by VMware performance expert Jeff Buell, which looks in detail at the relative performance of a bare-metal 32-node Hadoop cluster compared to a range of virtual clusters with up to 128 VMs. The executive summary is that while we saw a 13% performance degradation in a head-to-head comparison of a 32-node physical cluster against a 32-VM virtual cluster (one VM per host) running on the same hardware and running the same tests, virtualized performance can be increased significantly — to the point where virtualized Hadoop actually runs a bit faster than physical — by increasing the number of VMs per host. We’ve seen this effect before with Hadoop and with other resource-intensive HPC applications.

Read the full paper for detailed results and to learn about performance best practices for deploying Hadoop on vSphere.



Other posts by

High Performance Computing Conference: VMware at SC’17 in Denver next week

VMware will once again have a demo kiosk inside of the Dell EMC booth at Supercomputing, which is being held next week in Denver. We will showcase the benefits of virtualization of on-premise HPC environments with good performance for an array of workloads. We have a lot to talk about! Here is a preview… New […]

vSphere Scale-Out for HPC and Big Data

I’m very excited that we’ve announced vSphere Scale-Out this week at VMworld here in Las Vegas. This new vSphere edition is specifically and exclusively designed for running HPC and Big Data workloads. This is an important development in our work to offer compelling virtualization solutions for these two emerging workload classes. Our strategy for addressing […]

Three Extreme Performance Talks from the Office of the CTO at VMworld USA

The Office of the CTO will be presenting three talks in the unofficial “Extreme Performance” series at the upcoming VMworld 2017 conference in Las Vegas. In addition, one of these talks will be delivered at VMworld Europe in Barcelona. Each of these talks focuses on important aspects of pushing the envelope to achieve high performance […]