Introducing Phat, an Asynchronous Rails app

Phat is my new Rails 2.3.5 application which runs 100% asynchronous, supporting many concurrent requests in a single Ruby process.

This is a new breed of Rails application which uses a new mode of execution available in Ruby 1.9: single Thread, multiple Fiber. Existing modes of execution suck:

  • Single thread harkens back to the days of Rails 1.x, where you started N mongrels to handle up to N concurrent requests.
  • Multiple threads is better but still has fundamental issues in Ruby. Autoloading is simply broken and Ruby’s thread implementation does not scale at all due to the GIL.

Here’s a sample action which uses memcached and the database. There’s nothing odd here – it’s the same old Rails API and codebase we are used to as Ruby developers, it just executes differently under the covers.

class HelloController < ApplicationController
  def world
    site_ids = Rails.cache.fetch 'site_ids', :expires_in => 1.minute do
      Site.all.map(&:id)
    end
    render :text => site_ids
  end
end

How does it work? If you want the nitty-gritty, watch my talk on EventMachine and Fibers. Everything that does network access ideally should be modified to be Fiber-aware. I’ve updated many gems to be Fiber-aware: memcache-client, em_postgresql (and activerecord), cassandra, bunny and rsolr to name a few. You’ll also need to run thin as your app server, since all of this code assumes it is executing within EventMachine.

Additionally we need to ensure that each request runs in its own Fiber. My new gem, rack-fiber_pool, will do this for you, just add it as Rack middleware in config/environment.rb. Here’s the basic configuration:

# Asynchronous DNS lookup
require 'em-resolv-replace'
require 'rack/fiber_pool'
# Pull in the evented memcache-client.
# You'll need to configure config.cache_store as normal.
require 'memcache/event_machine'

Rails::Initializer.run do |config|
  config.cache_store = :mem_cache_store
  # Run each request in a Fiber
  config.middleware.use Rack::FiberPool
  # Get rid of Rack::Lock so we don't kill our concurrency
  config.threadsafe!
end

Additionally we need to configure Postgresql and disable ActionController’s reloader mutex as it really doesn’t like fibered execution. This is ok because remember – there’s only a single thread executing in our process!

With that done, we can try some tests to see how we scale now. EventMachine works best when you have significant network latency. A simple test with database access over coffeeshop WiFi:

Without EventMachine:
Requests per second: 4.39 [#/sec] (mean)

With EventMachine:
Requests per second: 21.31 [#/sec] (mean)

That’s it! There’s no magic here: you can make your Rails app a “phat” app by following the same guidelines above. Fire up one thin instance per processor/core, put nginx in front of it and it should scale like crazy!

24 thoughts on “Introducing Phat, an Asynchronous Rails app”

  1. Sounds great! Thanks for the article!
    So on the server side we can have only one working process of thin, and that’s all?? And what about 1.8.x apps, is there a way to archive the same behaviour of request handling?

  2. I would recommend one thin process per processor/core. Because there is only one underlying thread, you can only peg one core at a time. Basically think of thin as Mongrel, except it can handle many concurrent requests safely.

    Fibers are only in 1.9. There has been some work done to port them to 1.8 but I strongly urge just upgrading. You also gain a nice improvement in Ruby execution speed.

  3. And what about rails3? Do all the gems (eventmachine, rack-fiber_pool, em-resolv-replace, memcache-client) play well with new rails? And we also need to disable action_controller mutex and config.threadsafe!, right? Mb someday you’ll create rails3 completely async example =)

  4. Yes, everything should work except em_postgresql.

    Since Rails3 is still under development and I need to perform some ActiveRecord monkeypatching, I wanted to stick with a stable target like Rails 2.3. Now that it is done, I may see what needs to be done for Rails3.

  5. Thread locals are also Fiber local in Ruby 1.9. You need to make sure that the FiberPool appears in the middleware chain before any Thread locals are set. I updated the rack-fiber_pool README a few days ago with this info.

  6. Hi Mike,

    I’ve been trying to use llya’s em-mysqlplus + your fiber connection pool to get a simple rails 3 app up and running. After a bit of tweaking I think I’m almost there.

    I’m trying to run rake db:migrate but it’s complaining that eventmachine isn’t initialized.

    So does this mean that I have to somehow initialize eventmachine when not running inside thin (i.e. in a script)?

    Any help would be appreciated…

    Thanks,

    Saimon

  7. Yes, EM must be started since we are using it to manage the asynchronous I/O. My first thought is that we really don’t need async for rake tasks and therefore the em-mysqlplus driver should work synchronously if EM.reactor_running? == false.

    There’s many hacks you can do to solve this issue but no clean solution yet.

  8. Hi Mike,
    great work you did here. This makes me more excited than Rails3!

    I have some problems with configuring postgresql when trying to re-run the ab benchmarks. I always get the error “sorry, too many clients already”. could you show your postgresql.conf in a gist or via a commit in the phat repo? That would be really appreciated…

  9. > Everything that does network access ideally should be modified to be Fiber-aware.

    Hey Mike,

    Does this include gems like juggernaut, which does network access but it can be done locally?

  10. @Mike, Love the app, watched the presentation (didn’t like when that guy compared lovely open source ruby to windows). Anyway, I’m wondering what the differences between rack-fiber_pool and neverblock or is neverblock essentially the same thing minus the em_memcache?

    @Gonçalo Silva, Juggernaut uses event machine to queue its messages, and runs in a separate process – so I’d say it doesn’t really need any modification.

  11. Hi, great article. I’m going to ask the same question I just asked on Ilya Grigorik’s website though :)

    Two years later, what is the state of affairs with regard to a fully asynchronous Rails (an easy to install and configure one). Having read a lot of the articles and blog posts about anything Fiber/EM/async/concurrent Rails-related in the past weeks, it seems that your work and Ilya’s are still state of the art as of today…

    Which says a lot about your vision, dedication and good work, but also underlines the lack of involvement from the Rails community as a whole to bring the issue of proper concurrency to the spotlights. Any thoughts?

    1. Let me be perfectly clear about where I stand on em-synchrony and Fibered EM today, after spending a year working with the technology:

      It’s one giant hack and should not be used.

      My concurrency work for the last year has all focused on threads and the use of Actors to make them simpler to manage. My girl_friday and sidekiq projects are multi-threaded and not asynchronous.

  12. hich says a lot about your vision, dedication and good work, but also underlines the lack of involvement from the Rails community as a whole to bring the issue of proper concurrency to the spotlights. Any thoughts?

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>