Infoworld Virtualization Report just had a news article on VMware relaxing their benchmarking policy, though it seems that publishing benchmark results still requires prior approval by VMware. The article states: “provided that VMware has reviewed and approved of the methodology, assumptions and other parameters of the study”.
Virtualization is a very hot topic in the IT industry, yet there is very little quantitative data about security, isolation, performance, scalability, density, manageability and other characteristics of the virtualization products. Conducting benchmarking tests is neither simple nor cheap, and we feel that existing and potential customers should be able to access results of such tests; and we stand by the performance edge that Virtuozzo offers.
In my opinion, a comprehensive virtualization benchmark is long due. The best solution would be for all virtualization players to commission an independent body to create a set of fair tests and then use it to benchmark their products. SPEC and TPC have been criticized, but they still remain some of the most useful and unbiased benchmarking suits. SPEC-VM could be a nice addition to the SPEC family.
Meanwhile, if you want to test Virtuozzo performance, just let me know. If you happen to have a VMware-approved methodology, please don’t hesitate to contact me to arrange for a competitive benchmarking of Virtuozzo.