Sidekiq – simple, efficient messaging for Rails

I introduced my latest project, Sidekiq, this morning. Sidekiq is the culmination of several years of research and development into message processing with Ruby. I first wrote a single-threaded mysql-backed queue called QBert for FiveRuns in 2008 before delayed_job was introduced. Next I wrote a single-threaded SQS-based processor called Jobber and then a Fiber-based, RabbitMQ-backed processor called Qanat for OneSpot. Most recently I wrote girl_friday, an Actor-based, redis-backed processor and a custom multi-threaded version of Resque for a Carbon Five client.

So yeah, messaging and I have a history but I think that’s because Ruby still doesn’t have a simple, scalable option for message processing. Sidekiq is my attempt to solve that need.

Simple

Sidekiq integrates tightly with Rails 3 so your application’s worker code is nearby and easy for Sidekiq to find and load. I’d like to support plain Ruby applications but I need a replacement for the nice integration Rails 3 gives me.

Efficient

Sidekiq uses multiple threads so you can process hundreds of messages in parallel if you desire without the overhead of hundreds of processes. Internally Sidekiq uses the Celluloid actor library to make threading easier and safer to manage.

Developer Friendly

I’ve tried to make Sidekiq developer friendly in two ways:

  • Resque-compatible – I love conventions, why reinvent the wheel when you can stick closely to what someone else has done and leverage that work? I aim to make Sidekiq easy to integrate into an existing Resque project and monitorable via resque-ui.
  • Middleware – Sidekiq pulls in a middleware API similar to Rack for running code before/after/around a processed message and ships with middleware that helps with ActiveRecord and Airbrake.

Please let me know how you like Sidekiq – I love hearing stories about how people are using it.

6 thoughts on “Sidekiq – simple, efficient messaging for Rails”

  1. Your licensing is certainly not open source (but I guess this is the intention). How your additional restriction works with sections 7 and 10 I have no idea. For example: “All other non-permissive additional terms are considered “further restrictions” within the meaning of section 10. If the Program as you received it, or any part of it, contains a notice stating that it is governed by this License along with a term that is a further restriction, you may remove that term.”
    GPL-3 is not meant to be used in this way.

  2. Sorry for the judgment from my bell tower, but you killing a good project with a bad license.
    Most of the people in the community will prefer to contribute to the worst, but more free project. At least for me, it doesn’t make sense to contribute to the library, which i can’t modify without your permission.

  3. GPL? I’m sorry, but as much as this project looks promising, I’m not touching anything licensed differently than ruby/rails ecosystem’s de-facto-standard MIT license.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>