This is a new breed of Rails application which uses a new mode of execution available in Ruby 1.9: single Thread, multiple Fiber. Existing modes of execution suck:

• Single thread harkens back to the days of Rails 1.x, where you started N mongrels to handle up to N concurrent requests.
• Multiple threads is better but still has fundamental issues in Ruby. Autoloading is simply broken and Ruby’s thread implementation does not scale at all due to the GIL.

Here’s a sample action which uses memcached and the database. There’s nothing odd here – it’s the same old Rails API and codebase we are used to as Ruby developers, it just executes differently under the covers.

class HelloController < ApplicationController
  def world
    site_ids = Rails.cache.fetch 'site_ids', :expires_in => 1.minute do
      Site.all.map(&:id)
    end
    render :text => site_ids
  end
end

How does it work? If you want the nitty-gritty, watch my talk on EventMachine and Fibers. Everything that does network access ideally should be modified to be Fiber-aware. I’ve updated many gems to be Fiber-aware: memcache-client, em_postgresql (and activerecord), cassandra, bunny and rsolr to name a few. You’ll also need to run thin as your app server, since all of this code assumes it is executing within EventMachine.

Additionally we need to ensure that each request runs in its own Fiber. My new gem, rack-fiber_pool, will do this for you, just add it as Rack middleware in config/environment.rb. Here’s the basic configuration:

# Asynchronous DNS lookup
require 'em-resolv-replace'
require 'rack/fiber_pool'
# Pull in the evented memcache-client.
# You'll need to configure config.cache_store as normal.
require 'memcache/event_machine'
 
Rails::Initializer.run do |config|
  config.cache_store = :mem_cache_store
  # Run each request in a Fiber
  config.middleware.use Rack::FiberPool
  # Get rid of Rack::Lock so we don't kill our concurrency
  config.threadsafe!
end

Additionally we need to configure Postgresql and disable ActionController’s reloader mutex as it really doesn’t like fibered execution. This is ok because remember – there’s only a single thread executing in our process!

With that done, we can try some tests to see how we scale now. EventMachine works best when you have significant network latency. A simple test with database access over coffeeshop WiFi:

Without EventMachine:
Requests per second: 4.39 [#/sec] (mean)

With EventMachine:
Requests per second: 21.31 [#/sec] (mean)

That’s it! There’s no magic here: you can make your Rails app a “phat” app by following the same guidelines above. Fire up one thin instance per processor/core, put nginx in front of it and it should scale like crazy!

Now that I have RabbitMQ, Cassandra, Solr and the Amazon AWS services evented, the only holdup was ActiveRecord. Some people may advocate using another ORM layer but when you have 2-3 other Rails apps, all sharing 100+ models, you can’t afford to maintain two separate ORM layers. Plus, frankly I like the Rails stack: it works pretty well, is thoroughly documented and every Ruby developer is familiar with it.

So what do we need to do to get AR working event-style? At a high level, there’s two things required:

• The database driver itself must be modified to send SQL asynchronously. The postgresql driver, for instance, calls the exec(sql) method for all traffic to the database. So we just need to provide an exec method which uses Fibers under the covers to work asynchronously.
• AR’s connection pooling needs to be Fiber-safe. Out of the box, it is Thread-safe. Since we are using an execution model based on a single Thread with multiple Fibers, all the Fibers would try to use the same connection, with disastrous consequences.

These are the things that em_postgresql does.

• postgres_connection is a basic, EM-aware Postgres driver. It provides the Fibered exec() method which makes the whole thing asynchronous.
• em_postgresql_adapter.rb wraps postgres_connection to make it a proper ActiveRecord driver.
• patches.rb overrides a bunch of AR’s internal connection pooling to make it Fiber-friendly.

Unfortunately the latter makes one hack necessary – we have to have a list of current Fibers to release any lingering connections associated with those Fibers. The Threaded version can use Thread.list but Ruby does not provide an equivalent method for Fibers. Instead I require the application to register a FiberPool with AR to clear stale connections.

So what does it all mean? Well, here’s a Sinatra application that uses plain old ActiveRecord and is completely asynchronous! Try ab -n 100 -c 20 http://localhost:9292/test to hit the app with 20 concurrent connections; it will process them all in parallel, without any painful threading issues (autoloading, misbehaving extensions, etc). Awesome!

You should guess what’s next. Coming soon: the whole Rails stack, running asynchronously…

DataFabric is a Rails plugin/gem that adds sharding support to ActiveRecord. It supports Rails 2.x and is theoretically database independent, although I haven’t tested it personally with anything but MySQL and SQLite. Please see the github home page for the latest code, examples and a more in-depth explanation.

Rails Engines (PPT, 2.9MB)
Rails Engines (Keynote, 1.8MB)

I performed the brain surgery to use 1.7.4 (since we are using Rails 2.1) and deployed. See if you can guess when. Graph shows operations per hour (more is better).

config/initializers/rails_patches.rb

Rails.cache.instance_eval <<-EOM
  def read_multi(*keys)
    @data.get_multi(*keys)
  end
EOM

Rails uses read/write for its API naming so we name the method read_multi rather than get_multi. Here’s a sample usage in script/console:

>> Rails.cache.write('a', 1)
>> Rails.cache.write('b', 2)
>> Rails.cache.write('c', 3)
>> Rails.cache.read_multi('a', 'b', 'c')
=> {"a"=>1, "b"=>2, "c"=>3}

Enjoy!

Let’s spelunk through the code:

rails-2.3.2/lib/rails/plugin/loader.rb

        def configure_engines
          if engines.any?
            add_engine_routing_configurations
            add_engine_controller_paths
            add_engine_view_paths
          end
        end
 
        def add_engine_routing_configurations
          engines.select(&:routed?).collect(&:routing_file).each do |routing_file|
            ActionController::Routing::Routes.add_configuration_file(routing_file)
          end
        end
 
        def add_engine_controller_paths
          ActionController::Routing.controller_paths += engines.collect(&:controller_path)
        end
 
        def add_engine_view_paths
          # reverse it such that the last engine can overwrite view paths from the first, like with routes
          paths = ActionView::PathSet.new(engines.collect(&:view_path).reverse)
          ActionController::Base.view_paths.concat(paths)
          ActionMailer::Base.view_paths.concat(paths) if configuration.frameworks.include?(:action_mailer)
        end

For each engine, we add any routes, any controllers and any views. Additionally, the directories within app will be added to the global LOAD_PATH, as with a normal application. Note that engines are processed in order exactly like plugins: alphabetically or based on the order they are listed in config/environment.rb.

There are some limitations you should be aware of:

• No migration support: while the engine can add models, it is not obvious how to manage any database structure needed by the engine. I would imagine the engine should use the install.rb hook to copy migrations to the app’s db/migrate directory.
• No public asset support: like migrations, any stylesheets, javascripts or images must be copied as part of the install.rb hook to the app’s public directory.
• Like plugins, naming becomes a concern. An engine can have a User model but this will lead to problems with the 90% of Rails applications that have a model of the same name. You can put your models within a module but I’ve heard of problems when trying to mix Rails autoloading with modularized classes. As with plugins, be sure to err on the side of safety and use a unique name for your classes. I’m building an engine called Queso and it provides a model called QuesoSearch, which is unlikely to collide with application classes unless you are building an application for a Mexican cheese provider.

So while Engines do have some limitations to be aware of, they do fill a valuable niche; engines provide a good framework for building full-stack generic application functionality. ActiveScaffold is one example of a Rails plugin that would be an excellent choice to rewrite as an Engine.

I was a little unhappy with my wrapup – the one thing I wanted to teach people was when to use each different caching mechanism provided by Rails and I didn’t really revisit and summarize that content. So here’s a quick summary:

• HTTP caching – prefer this over all other mechanisms. This is really the only mechanism that prevents the request from ever hitting Ruby. This topic is big enough for a book so I won’t cover it here but review the Expires, Etag and Cache-Control headers to understand how HTTP caching works. You’ll need to configure Varnish, Squid, mod_cache or some other HTTP caching proxy.
• Page caching – I believe this is really legacy from before Rails supported HTTP caching properly. Stick with HTTP caching and proper headers.
• Action caching – useful when the entire page contents can be cached but you need to run before_filters (e.g. to ensure the user is logged in). Use AJAX/javascript to do minor customization to the cached content.
• Fragment caching – useful when various boxes of content on the page can be cached, but have different dependencies and need to be expired at different times
• Object caching (the Rails.cache.fetch method) – the most granular mechanism. Good for caching the results of intensive logic or queries.

I hope this helps demystify the myriad of caching mechanisms Rails supports. If you want to learn even more, Gregg Pollack has an amazing set of videos on Scaling Rails which covers caching in great depth. Happy Caching!

require 'rubygems'
 
# Brain surgery to use our own version of memcache-client without
# having to modify activesupport directly.
# Unload any previous instance of the class
if Object.const_defined? :MemCache
  Object.instance_eval { remove_const :MemCache }
end
# Pull in the exact version we want
gem 'memcache-client', '1.6.5'
# Ensure that the memcache-client path is at the front of the loadpath
$LOAD_PATH.each do |path|
  if path =~ /memcache-client/
    $LOAD_PATH.delete(path)
    $LOAD_PATH.unshift(path)
  end
end
# If Ruby thinks it's already loaded memcache.rb, force
# a reload otherwise just require.
if $".find { |file| file =~ /\Amemcache.rb\Z/ }
  load 'memcache.rb'
else
  require 'memcache'
end

If someone knows of a cleaner way to do this, please let me know.

1) Request an invite by signing up at https://dash.fiveruns.com
2) Log into Dash, create a new Rails application and note your application token.
3) Install the gem:

sudo gem install json
sudo gem install fiveruns-dash-rails -s http://gems.github.com

4) Add the gem to your config/environment.rb

Rails::Initializer.run do |config|
  config.gem 'json' # needed because of a bug in json_pure
  config.gem 'fiveruns-dash-rails',
             :lib => 'fiveruns_dash_rails',
             :source => 'http://gems.github.com'

5) Add the initializer file config/initializers/dash.rb to start Dash:

if defined?(Fiveruns::Dash)
  Fiveruns::Dash::Rails.start(:production => 'YOUR-APP-TOKEN-HERE')
end

This will start Dash in the production environment. We don’t recommend using Dash in development mode. If you have a staging environment, you can create a separate app for that environment and combine the two like so:

if defined?(Fiveruns::Dash)
  Fiveruns::Dash::Rails.start(:production => 'PROD-TOKEN', :staging => 'STAGE-TOKEN')
end

Email me if you have problems – I’m happy to help people give Dash a test drive. Dash is 100% free right now until we’ve worked out pricing models but rest assured there will always be a free tier so you don’t have to worry about your app costing you money in the future. I hope your Dash test drive is full of rimshots with no sad trombones!