VMware® VMmark® has quickly become a performance benchmark to which many organizations turn to quantify how many virtual machines (VMs) they can realistically expect to host and then perform well on a cluster of physical servers. Yet a published VMmark score for a specified hardware configuration may overstate or, conversely, fail to fully reflect the particular solution’s VM consolidation and performance capabilities. The HP ProLiant BL660c published VMmark performance benchmarks using a backend HP 3PAR StoreServ 7450 all-flash array provide the relevant, real-world results that organizations need to achieve maximum VM density levels, maintain or even improve VM performance as they scale and control costs as they grow.
VMware VMmark Begins the Transformation of Virtualization Performance Measurement from an Art into a Science
Server virtualization is becoming, and largely has already become, the de facto way in which organizations introduce new applications into their IT infrastructure. Yet the techniques associated with forecasting the right number of VMs to host on each physical machine remains as much of an art as it is a science due to the variables involved (application type, server and storage hardware , etc.) that influence performance in a virtualized environment.
The VMware VMmark performance benchmark mitigates some of this guesswork associated with determining how many VMs a particular configuration can host. Introduced in 2007, the VMware VMmark virtualization platform scores performance in virtualized environments with mixed workloads by grouping VMs into tiles (the metric used by VMmark to benchmark virtualization performance,) measuring how quickly each tile completes specific tasks and then averaging each tile’s scores to provide an overall score.
The current VMmark 2.X increased the number of VMs in a tile from 6 to 8 and included additional tools for benchmarking multi-server virtualization environments. For example, VMmark 2.x measures the performance impacts of virtualization-specific workloads such as vMotion and storage vMotion on the underlying platform. The recent VMmark 2.5.x even provides specifications to measure power usage in virtualized environments.
These iterations of the VMware VMmark performance benchmark have collectively solved the following challenges for organizations by enabling them to:
- Objectively quantify how many VMs a physical machine can provide a sufficient, reliable amount of performance to host.
- View and compare the performance of different hardware platforms based upon published metrics
- Better forecast and budget for the appropriate amount of hardware to host their VMs
The Storage Variable
Despite the standardization that the VMmark performance benchmark has introduced into the previously subjective areas of measuring performance and power utilization in virtualized environments, variables still exist in how tested hardware solutions may be configured.
An allowance in the hardware configuration that the VMmark specification makes is permitting the use of various storage configurations when running these benchmarks. While this may seem like a nit, storage is often one of the leading contributors to performance bottleneck in many virtualized environments. As such, using a storage configuration that does not reflect a real world environment impacts how one should interpret published VMmark performance results. Two specific conditions to examine in these performance results to understand their relevance in the real world include:
- Storage configuration may not reflect real-world production conditions. Organizations often use external storage arrays to host production VMs. However many of these VMmark benchmark tests are conducted using either internal flash drives or external storage arrays with their flash or hard disk drives configured as RAID 0. While these storage configurations contribute to improved performance for the purposes of the VMmark performance benchmark, they do not represent what most organizations use in their production environments.
- Storage array performance capabilities not fully measured. The intent of the VMmark performance benchmark is to provide guidance as to the number of VMs with mixed workloads to which the server hardware can provide sufficient and reliable performance. While the storage array contributes to and influences performance, VMmark does not attempt to examine the specific performance capabilities of the underlying storage array. As such, it is both possible and probable that the full performance capabilities of the storage array are not fully tapped.
The Distinguishing Characteristics of the HP ProLiant VMmark Benchmarks
One of the notable exceptions to the many published VMmark benchmarks is the recent April 15, 2014, disclosure of the HP ProLiant BL660c Gen8 VMmark benchmarks. These HP ProLiant VMmark benchmarks distinguish themselves from other VMmark benchmark results in the following three ways:
- HP 3PAR StoreServ 7450 utilized in the benchmarking. The HP 3PAR StoreServ 7450 is recognized by DCIG at a Top 3 enterprise all-flash storage array and is routinely used by organizations in their production environments. By using the HP 3PAR StoreServ 7450 as part of the HP ProLiant BL660c Gen8 VMmark benchmark, the published results provide a more realistic representation of the high levels of performance and VM density that an organization could expect of the HP ProLiant BL660c if deployed in their production environment since it uses a production storage array.
- RAID 1 storage configuration used on the HP 3PAR StoreServ 7450. Almost all organizations without exception put the disk drives in their production storage array into some type of RAID configuration for data protection. In the HP VMmark benchmarks, a RAID 1 configuration was used for all of the tests as opposed to a RAID 0 configuration. This RAID 1 configuration more accurately represents the type of RAID configuration that would be found and used in a production storage array deployment.
- More HP ProLiant BL660c Gen8 servers could be introduced, use the same HP 3PAR StoreServ 7450 and achieve comparable VM density and performance results. The published April 15, 2014, HP ProLiant BL660c VMmark benchmarks only include the performance results of two HP ProLiant hosts. These results suggest that an organization could potentially host up to 192 VMs on these two blade servers with each of these 192 VMs having access to very high levels of performance.
However there is nothing in these results to suggest that these two HP ProLiant servers were in any way close to maxing out the performance capabilities of the HP 3PAR 7450 array. This gives organizations the flexibility to connect more physical HP ProLiant BL660c servers to the HP 3PAR StoreServ 7450 array with the knowledge that they can achieve approximately the same levels of VM density and performance on these additional ProLiant servers as they did on their first two ProLiant servers without needing to acquire additional storage.
HP ProLiant BL660c Published VMmark Benchmarks Answer Call for Performance Tests that Have Real World Application
To say that data centers in general and virtualized data centers specifically need meaningful performance benchmarks in order to make better capacity planning and buying decisions is an understatement. In that sense, the VMware VMmark performance benchmarks provide an important step forward in providing organizations with the information they need to better assess and optimally utilize the hardware intended for hosting VMs in their environment.
However any performance benchmark and its application in production virtualized environments is only as useful as the underlying hardware configuration on which that benchmark is based. The most recent HP ProLiant BL660c Gen8 VMmark benchmarks provide these sought after real-world performance metrics that enterprise virtualized environments want and need. While all organizations should always test new solutions in their environment before deploying them if at all possible, the use of the HP 3PAR StoreServ 7450 in these published VMmark benchmarks give organizations more reason than normal to be optimistic about the potential VM density and performance benefits that the 7450 can realistically deliver in their environment.