One email I get fairly regularly is “Why would I use memcache-client when the performance sucks?” Most people point me to Evan Weaver’s BENCHMARKS file in his memcached library as proof that I should just close up shop.
His benchmarks, like all performance tests, are a snapshot in time – they reflect a certain revision of the codebase. Specifically I assume his benchmarks reflect memcache-client 1.5.0. This version is about 18 months old and ships with Rails 2.x. Since 1.5.0, I’ve taken over the project and done a lot of tuning. Here’s some numbers:
Testing 1.6.4
user system total real
set:plain:memcache-client 0.740000 0.270000 1.010000 ( 2.123806)
set:ruby:memcache-client 0.800000 0.270000 1.070000 ( 2.215323)
get:plain:memcache-client 0.920000 0.270000 1.190000 ( 2.270049)
get:ruby:memcache-client 1.000000 0.260000 1.260000 ( 2.473615)
multiget:ruby:memcache-client 0.600000 0.090000 0.690000 ( 1.055913)
missing:ruby:memcache-client 0.710000 0.260000 0.970000 ( 2.069232)
mixed:ruby:memcache-client 1.800000 0.540000 2.340000 ( 4.637675)
Testing 1.5.0 (as shipped in Rails 2.2.2)
user system total real
set:plain:memcache-client 28.450000 0.400000 28.850000 ( 30.025903)
set:ruby:memcache-client 29.180000 0.420000 29.600000 ( 30.803760)
get:plain:memcache-client 28.640000 0.410000 29.050000 ( 30.201306)
get:ruby:memcache-client 29.350000 0.430000 29.780000 ( 30.942459)
multiget:ruby:memcache-client 30.750000 0.250000 31.000000 ( 31.462482)
missing:ruby:memcache-client 28.470000 0.410000 28.880000 ( 30.046172)
mixed:ruby:memcache-client 58.590000 0.840000 59.430000 ( 61.951933)
You can see the exact benchmark code in memcache-client/test/test_benchmark.rb. It’s basically just a direct copy of Evan’s benchmarks, modified to run as part of the test suite. This is running in Ruby 1.8.6 p114 on a 2.2Ghz Core 2 Duo. The numbers for Ruby 1.9.1 are approx 20% faster.
The latest numbers are not orders of magnitude slower but rather merely some percentage off. So while Evan’s numbers are correct for an old version of memcache-client, they do not represent the current state of the project. It’s my belief that memcache-client 1.6.x has perfectly acceptable performance and in return, you get a mostly pure Ruby codebase that works on JRuby, Ruby 1.8 and Ruby 1.9. I hope this clears up any confusion; let me know if you have further questions!
4 responses so far ↓
1 Radarek // Feb 17, 2009 at 11:37 am
Results don’t say to much without comparing to other memcached library.
2 mperham // Feb 17, 2009 at 12:23 pm
I could not get the memcached gem to install on OSX. The instructions in the README simply don’t work.
It doesn’t matter how fast something is if it doesn’t install.
3 GMFlash // Feb 24, 2009 at 1:08 am
The 0.14/0.26 version of the memcached gem doesn’t work. Evan recommended in the support forum to use the Ruby client and C library from this page: http://blog.evanweaver.com/articles/2009/01/24/secret-codes/
Hope this works so we can see how they stack up. Having similar or better performance without compiling a lib/gem would be great!
4 rmxz // Oct 11, 2009 at 3:37 pm
In addition to the absolute performance, another interesting comparison is how much faster it is to fetch data from memcached vs fetching the objects from an underlying database.
On one of my apps, the pure-ruby memcached library was no faster than storing and fetching objects directly ton an from a postgres database.
OTOH, the native memcached client produced factor-of-3 improvements in the speed of the slowest html pages.
Leave a Comment