Introducing Phat, an Asynchronous Rails app

April 3rd, 2010 · 20 Comments

Phat is my new Rails 2.3.5 application which runs 100% asynchronous, supporting many concurrent requests in a single Ruby process.

This is a new breed of Rails application which uses a new mode of execution available in Ruby 1.9: single Thread, multiple Fiber. Existing modes of execution suck:

Single thread harkens back to the days of Rails 1.x, where you started N mongrels to handle up to N concurrent requests.
Multiple threads is better but still has fundamental issues in Ruby. Autoloading is simply broken and Ruby’s thread implementation does not scale at all due to the GIL.

Here’s a sample action which uses memcached and the database. There’s nothing odd here – it’s the same old Rails API and codebase we are used to as Ruby developers, it just executes differently under the covers.

class HelloController < ApplicationController
  def world
    site_ids = Rails.cache.fetch 'site_ids', :expires_in => 1.minute do
      Site.all.map(&:id)
    end
    render :text => site_ids
  end
end

How does it work? If you want the nitty-gritty, watch my talk on EventMachine and Fibers. Everything that does network access ideally should be modified to be Fiber-aware. I’ve updated many gems to be Fiber-aware: memcache-client, em_postgresql (and activerecord), cassandra, bunny and rsolr to name a few. You’ll also need to run thin as your app server, since all of this code assumes it is executing within EventMachine.

Additionally we need to ensure that each request runs in its own Fiber. My new gem, rack-fiber_pool, will do this for you, just add it as Rack middleware in config/environment.rb. Here’s the basic configuration:

# Asynchronous DNS lookup
require 'em-resolv-replace'
require 'rack/fiber_pool'
# Pull in the evented memcache-client.
# You'll need to configure config.cache_store as normal.
require 'memcache/event_machine'
 
Rails::Initializer.run do |config|
  config.cache_store = :mem_cache_store
  # Run each request in a Fiber
  config.middleware.use Rack::FiberPool
  # Get rid of Rack::Lock so we don't kill our concurrency
  config.threadsafe!
end

Additionally we need to configure Postgresql and disable ActionController’s reloader mutex as it really doesn’t like fibered execution. This is ok because remember – there’s only a single thread executing in our process!

With that done, we can try some tests to see how we scale now. EventMachine works best when you have significant network latency. A simple test with database access over coffeeshop WiFi:

Without EventMachine:
Requests per second: 4.39 [#/sec] (mean)

With EventMachine:
Requests per second: 21.31 [#/sec] (mean)

That’s it! There’s no magic here: you can make your Rails app a “phat” app by following the same guidelines above. Fire up one thin instance per processor/core, put nginx in front of it and it should scale like crazy!

Tags: Rails

20 responses so far ↓

1 Mikel Lindsaar // Apr 3, 2010 at 6:40 pm

If this does what you say it does, that is very awesome! I’ll have to give it a shot.
2 Daniel Huckstep // Apr 4, 2010 at 3:41 am

This is totally cool. I wish I had a rails app to use it in…
3 Ivan Ukhov // Apr 5, 2010 at 5:33 am

Sounds great! Thanks for the article!
So on the server side we can have only one working process of thin, and that’s all?? And what about 1.8.x apps, is there a way to archive the same behaviour of request handling?
4 Mike Perham // Apr 5, 2010 at 10:00 am

I would recommend one thin process per processor/core. Because there is only one underlying thread, you can only peg one core at a time. Basically think of thin as Mongrel, except it can handle many concurrent requests safely.

Fibers are only in 1.9. There has been some work done to port them to 1.8 but I strongly urge just upgrading. You also gain a nice improvement in Ruby execution speed.
5 Ivan Ukhov // Apr 5, 2010 at 2:32 pm

And what about rails3? Do all the gems (eventmachine, rack-fiber_pool, em-resolv-replace, memcache-client) play well with new rails? And we also need to disable action_controller mutex and config.threadsafe!, right? Mb someday you’ll create rails3 completely async example =)
6 Mike Perham // Apr 5, 2010 at 2:41 pm

Yes, everything should work except em_postgresql.

Since Rails3 is still under development and I need to perform some ActiveRecord monkeypatching, I wanted to stick with a stable target like Rails 2.3. Now that it is done, I may see what needs to be done for Rails3.
7 InVisible Blog » links for 2010-04-06 // Apr 6, 2010 at 11:03 am

[...] 6th, 2010 Introducing Phat, an Asynchronous Rails app (tags: rails ruby asynchronous performance rubyonrails) michaeldv's awesome_print at master [...]
8 Ash McKenzie // Apr 6, 2010 at 6:29 pm

Wow, definitely sacrificing some sleep tonight to convert over an app or two Thanks!
9 links for 2010-04-11 « Bloggitation // Apr 12, 2010 at 1:08 am

[...] Introducing Phat, an Asynchronous Rails app (tags: ruby rails rack eventmachine web programming) Categories: Microblog Comments (0) Trackbacks (0) Leave a comment Trackback [...]
10 Mick Staugaard // Apr 15, 2010 at 1:36 pm

How does code that depend on thread local variables behave? I’m thinking Time.zone, I18n.locale etc.
11 Mike Perham // Apr 15, 2010 at 2:00 pm

Thread locals are also Fiber local in Ruby 1.9. You need to make sure that the FiberPool appears in the middleware chain before any Thread locals are set. I updated the rack-fiber_pool README a few days ago with this info.
12 Non-blocking ActiveRecord & Rails - igvita.com // Apr 15, 2010 at 3:39 pm

[...] one, and finally, we need to disable the built in Mutex (hap tip to Mike Perham for doing all the dirty work for us). Now let's give it a [...]
13 Saimon Moore // Apr 16, 2010 at 9:01 am

Hi Mike,

I’ve been trying to use llya’s em-mysqlplus + your fiber connection pool to get a simple rails 3 app up and running. After a bit of tweaking I think I’m almost there.

I’m trying to run rake db:migrate but it’s complaining that eventmachine isn’t initialized.

So does this mean that I have to somehow initialize eventmachine when not running inside thin (i.e. in a script)?

Any help would be appreciated…

Thanks,

Saimon
14 Mike Perham // Apr 16, 2010 at 9:24 am

Yes, EM must be started since we are using it to manage the asynchronous I/O. My first thought is that we really don’t need async for rake tasks and therefore the em-mysqlplus driver should work synchronously if EM.reactor_running? == false.

There’s many hacks you can do to solve this issue but no clean solution yet.
15 Steven // Apr 26, 2010 at 10:00 am

Hi Mike,
great work you did here. This makes me more excited than Rails3!

I have some problems with configuring postgresql when trying to re-run the ab benchmarks. I always get the error “sorry, too many clients already”. could you show your postgresql.conf in a gist or via a commit in the phat repo? That would be really appreciated…
16 Gonçalo Silva // May 13, 2010 at 7:00 pm

> Everything that does network access ideally should be modified to be Fiber-aware.

Hey Mike,

Does this include gems like juggernaut, which does network access but it can be done locally?
17 Losty Otaku // May 28, 2010 at 6:45 am

@Mike, Love the app, watched the presentation (didn’t like when that guy compared lovely open source ruby to windows). Anyway, I’m wondering what the differences between rack-fiber_pool and neverblock or is neverblock essentially the same thing minus the em_memcache?

@Gonçalo Silva, Juggernaut uses event machine to queue its messages, and runs in a separate process – so I’d say it doesn’t really need any modification.
18 Ruby 1.9 Fibers + EventMachine for Big Ruby Webapp Performance Gains // Jun 14, 2010 at 1:10 am

[...] in April, Mike Perham introduced Phat, an asynchronous Rails 2.3.5 app running on Ruby 1.9 and supporting "many concurrent requests in a [...]
19 Особое программирование » Post Topic » Ruby 1.9 Fibers + EventMachine for Big Ruby Webapp Performance Gains // Jun 14, 2010 at 2:56 pm

[...] in April, Mike Perham introduced Phat, an asynchronous Rails 2.3.5 app running on Ruby 1.9 and supporting “many concurrent requests [...]
20 Co nowego w świecie Ruby’ego #4 // Jul 21, 2010 at 3:03 pm

[...] koncepcja i zaskakująco sprawne działanie. Jeden proces Thin’a, Fibers z Ruby 1.9 i skalowanie jak ta lala Znalezione przez forum [...]

Mike Perham

On Ruby, software and the Internet