Background Job Processing Overhead
I wrote a simple benchmark which loads 20,000 empty jobs into Redis and then times how long it takes one Sidekiq process to finish those jobs running on my 2012 MacBook Air. The times are unsurprising (lower is better):
JRuby 1.7.4 – 38 sec (using java 1.7.0_09)
MRI 2.0.0-p195 – 50 sec
MRI 1.9.3-p374 – 56 sec
Just as a point of comparison, I tried the latest version of Resque too:
MRI 2.0.0-p195 – 286 sec (Resque 1.24.1, one process)
Defaults were used where possible, I used Sidekiq 2.13.0 with 25 worker threads and the worker was a no-op class:
class LazyWorker include Sidekiq::Worker def perform end end
To reproduce the benchmark:
- Install redis.
- redis-cli flushall
- git clone git://github.com/mperham/sidekiq
- cd sidekiq/myapp
- bundle exec ruby bench.rb (this just loads 20k jobs into redis)
- bundle exec sidekiq
- Even with the GIL, MRI can perform 400 jobs/sec.
- In my experience, JRuby continues to be 20-30% faster than MRI in real world benchmarks. This was true a year ago with 1.6.x vs 1.9.3 and still true today with 1.7.x vs 2.0.0.
- I didn’t try Rubinius 2.0.0-dev because rbenv installs it in Ruby 1.8 mode (what year is this!?).