You are invited to a whimsical evening of merriment and gamesmanship!
What: Board Games and Beverages
Where: Skyroom Lounge, top of Red Lion Hotel, across the street from Railsconf
When: Tuesday, 4/30 7-11pm
The Skyroom has a great view of Portland so try to get there before sundown. We should have enough games for everyone but bring your favorite if you’d like. They have food if you need a late dinner. They have room for 250 so I’m hoping we’ll have plenty of room for everyone.
If you’re in, please comment to RSVP!
Want to know what’s going on during Railsconf 2013 outside of the conference center? Here’s a list of all events open to the public, this page will be updated as I learn of more events. Email mperham at gmail with event details.
KidsCodeCamp, 10am-4pm, Multnomah Arts Center
PlanetArgon.com Gorge Hike, 12:30-6:30pm, meet at Convention Center
Tech Women’s Communities Social, 7-9pm, NewRelic HQ
Code Fellows Happy Hour, 7:30-9:30pm, Rontoms
ZenCoder and SendGrid Party, 7:30-9:30pm, Ground Kontrol
Pivotal Labs Mixer, 9-11pm, Spirit of ’77
RailsConf Run 1, 7am, Steel Bridge
Board Games and Beverages, 7-11pm, Skyroom Lounge
A Night in the Ruby Sky, 7:30-10:30pm, NewRelic HQ
Railsconf Bike Ride, 6:45pm
CodeClimate Drinkup, 6pm, Green Dragon
Kickstarter Meetup, 7pm, Dig a Pony
EngineYard / Elemental After Party, 7:30-10pm, Elemental Office
Lumosity Party (pw: lumos), 8:30-11:30pm, Spirit of ’77
RailsConf Run 2, 7am, MAX Zoo Station
A user recently ran Sidekiq with Ruby 2.0 and found that the signal handling did not work well at all. Ctrl-C and other signals resulted in some ominous stack traces.
It turns out that Ruby 2.0 locks down what you can do in a signal handler in order to prevent unsafe or possibly non-deterministic behavior. You can’t take a Mutex within a signal handler anymore as this could result in a thread context switch or even deadlock. In fact you can’t even write to a Logger because it tries to use a Mutex internally.
I rewrote the signal handling to conform with the new restrictions: all handlers now just push the name of the signal onto a global array and the main thread polls once per second for unhandled signals. This isn’t perfect, polling is something to be avoided where possible, but I don’t know of a better solution and it’s a lot better than the previous “fat” handlers that did a lot of work.
Some back of the envelope numbers I was pondering this morning:
40 hours / week * 50 weeks / year = 2000 hr / yr
Bill Gates is worth about $66 billion. He’s worked for about 33 years now.
66 billion / 33 years = $2 billion/year / 2000 = $1 million / hour.
Bill Gates has made roughly $1 million per hour over the length of his career. Most people would be happy to make that amount in a decade. No moral or judgment here, just an astonishing number to think about.
This day last year I released Sidekiq 0.5.0 to the public. I left Carbon Five in mid-January and was moving to Portland to join The Clymb with wife and child in tow. For the previous few months, I’d been thinking about background processing and felt there was opportunity for a library that was more efficient and also provided more functionality. I spent my two weeks of downtime building what became Sidekiq.
I joined The Clymb one year ago today and what an amazing year it’s been.
Some context for you: I was employee 33 and the third engineer to join. The test suite had hundreds of failures and had been running red for months. We were on REE, Rails 2.3 and Delayed Job – a stack that was current two years before – and had no error service to know what errors were happening in production. The site was deployed with fabric, a Python-based tool. The first two developers were overworked and fought fires all day, every day.
Each year, Black Duck Software picks their top 10 new open source projects created that year. For 2012, Sidekiq has been selected one of those 10 OSS rookie projects! How cool is that?
I’ve spent the last few years working to advance and improve Ruby’s efficiency through concurrency, first with EventMachine and fibers and now with Actors and multithreading so it shouldn’t surprise you that my #1 pick is puma. It’s my belief that puma and sidekiq are a new breed of Ruby infrastructure that can dramatically improve your application’s efficiency — should you decide to take advantage of them.
puma is a pure Ruby, Rack-based web server and drops right in as a replacement for thin or unicorn. Unlike unicorn or thin, puma is designed to run multithreaded by default so you get far better memory efficiency. A typical single threaded Rails unicorn process takes 250MB. puma defaults to a maximum of 16 threads per process so one puma process can replace 16 unicorn processes taking 4GB of RAM! puma, like all multithreaded libraries, works best in a truly concurrent Ruby VM like JRuby or Rubinius but you’ll still get a big win running on MRI.
To test this, I ran 50 concurrent requests 20 times for a total of 1000 requests against a non-trivial endpoint on TheClymb.com Rails application. config.threadsafe! was enabled, a database pool size of 10 and puma’s default of 16 threads. Each request makes two database queries and renders a slim-based template.
Unicorn/MRI 1.9.3 is the baseline: single-threaded, it runs the 1000 requests in 19 seconds. Puma/MRI manages to speed up a bit but is still hampered by the GIL and runs in 15 seconds. Puma/JRuby unlocks the second core on my MacBook Air and runs in under 9 seconds!
What this means is simple: threading with puma will get you better performance than Unicorn, even on MRI, and jumping to JRuby gets you a significantly bigger boost by giving you truly parallel threads. It took me about one hour to get our Rails app, which has always run on MRI, working with JRuby. Give JRuby a try some weekend and you might be surprised how well it works!
I hope you enjoyed my 12 Gems of Christmas series and found a few gems that were worthy of further study.
ActiveRecord has been a huge boon for web development in promoting conventions in databases. Every new company I joined we had to determine how to name tables, primary keys and indexes. Standardizing id, automatic timestamps, pluralizing nouns, all of it makes development easier and more friendly to developers who just want to build stuff, not worry about every mundane detail.
All is not wine and roses though, ActiveRecord suffers from a common ORM performance issue: the N+1 query problem. Find all shopping carts created in the last month and print out the number of items in those carts. This is what you might see in your terminal:
select * from shopping_carts where created_at > 1.month.ago
select count(*) from shopping_cart_items where shopping_cart_id = 111
select count(*) from shopping_cart_items where shopping_cart_id = 222
select count(*) from shopping_cart_items where shopping_cart_id = 333
select count(*) from shopping_cart_items where shopping_cart_id = 444
select count(*) from shopping_cart_items where shopping_cart_id = 555
This is because the initial query loads the data associated with the shopping_carts but does not load the associated items. As you iterate through each cart, Rails lazy loads the item count; unfortunately it does it one cart at a time.
Enter bullet which aims to help you find and fix any N+1 query issues in your ActiveRecord code. Add it to your Gemfile and activate it in your Rails configuration, soon you’ll see warnings like this:
Unused Eager Loading detected
Brand => [:products]
Remove from your finder: :include => [:products]
As we learned when we were younger, knowing is half the problem. These performance problems should be easy to track down and tune now that you know about the problem; you’ll find your page rendering times plummet since the number of database queries performed will drop dramatically.
Tomorrow we’ll discuss my favorite subject, concurrency, and unveil the #1 gem!
Ever try to write a rake task that took one or more arguments? How about calling a rake task from another task? Rake makes basic invocation simple but everything else obtuse. Enter thor, courtesy of the unstoppable Yehuda Katz. Thor aims to make command line script development simpler by making your scripts standard Ruby objects and providing good documentation in its wiki for many common use cases. Consider a simple example, create a task to process a CSV file:
class Inventory < Thor
desc "process_csv FILE", "process the nightly inventory update"
method_option :delete, :aliases => "-d", :desc => "Delete the file after parsing it"
# do something, maybe like:
remove_file(file) if options[:delete]
You invoke the task via
thor inventory:process_csv some_file.csv. Tasks are just public methods within a subclass of Thor. The
method_option methods describe those tasks and any expected options on the command line.
Thor::Actions gives us some filesystem helper methods, like
remove_file. The wiki goes into further detail on things like inter-task dependencies, task grouping, namespaces, etc. Next time you need to write a command line script, give Thor a try and see if it doesn’t make your life easier.
Tomorrow, I’ll show you a tool for detecting and fixing a common Rails performance problem.