How Cloud A stacks up against the competition

We compared our performance using Redis-Benchmark next to some of the more traditional US based Cloud providers and this is what we found. Trying to keep the tests as “apples-to-apples” as possible, we chose the most similar instance sizes offered (1GB flavour) by each player and ran the tests using the same version of Ubuntu & Redis Server for each:

Cloud A Benchmarks:

Rackspace posted their benchmarks here: http://developer.rackspace.com/blog/redis-benchmark-and-rackspace-performance-vms.html

Digital Ocean Benchmarks:

Linode Benchmarks:

For Digital Ocean & Linode we created instances of the same size and the ones we tested for Cloud A. For the tests we ran, we took 5 samples for each test, removed the highest, lowest, and then took an average of the 3 remaining results.

What does a high benchmark score mean? The Redis team talks about the benchmarking tool here, and the factors that impact the requests per second the most are: virtualization overhead, per-core cpu performance, and RAM speed to name a few.

We are always measuring our own performance and taking queues from competitors on areas to optimize. We also realize that single benchmarks are to be taken with a grain of salt and we do not claim to be redis performance experts.

The best metric to consider is end user experience, in a recent video demo of how to create an instance it only took 24 seconds for Cloud A to instantiate a standard Windows VM (you can see it here) whereas Digital Ocean proudly advertises it takes 55 seconds to do the same think on their platform.

There will be more benchmarks posted soon around I/O performance, one of the biggest challenges in the public cloud space.