The Path to Full-time Open Source

Two years ago today I released Sidekiq Pro, my commercial extension to Sidekiq, as an experiment to see if I could make OSS development financially viable for individual developers. I had no idea if anyone would trust me and buy it. Can you think of anyone else selling a Rubygem?


Sidekiq Pro sales for the last three months of 2012 were $7500. In 2013 they totalled $85,000. This year sales should top $175,000. Open Source development is now my full-time job, with no need for a corporate patron!

I’m being financially transparent here because I want my fellow developers to know: you can build incredibly valuable software and make real money from it. A small percentage of your users are willing to pay for extra features and the assurance of ongoing support over the years. It does take time, experience and a little vision. Here’s what I did:

A Path to Success

  1. Find a tool/library that is non-trivial and an important component to your current system or workflow.
  2. Plan out how you can make it better: simplify it, discard superfluous functionality, add useful functionality.
  3. Divide the functionality into open source and commercial parts. Use a GNU license and a commercial license for the respective parts.
  4. Build the open source part, document it.
  5. If it takes off, build the commercial part and the infrastructure necessary to sell it.

This path will take many months and none of these steps are easy! Be patient and work part-time on it for as long as possible. Remember that the open source version itself needs to offer plenty of value for users or there’s no reason for them to use your creation. The most important aspect is that you are your own best customer: you know the current tool and the pain it causes. So as the old saying goes, build that better mousetrap!

For me, I started with Resque. It was a major component of so many Ruby on Rails websites, and yet it was barely maintained and missing a lot of important features. I knew I could build something better. From this “business plan”, I created Sidekiq and Sidekiq Pro by following the steps above.

Now I want to do it again.

Three months ago I quit my job to work on Sidekiq and build a brand new OSS project and commercial product. Tomorrow I want to introduce it to you.

2013-07-21 11.00.06c

Don’t Daemonize your Daemons!

For years developers have followed the same arcane dozen steps to create a long-lived daemon process on Unix-based systems. These steps were state of the art in 2000 but they are no longer best practice today. Jake Gordon’s recent blog post on daemonizing Ruby processes is 100% factual but his approach is not recommended these days. Your application code should not be dealing with PID files, log redirection or other low-level concerns.
Continue reading


Library Versioning

It’s time for our annual Semantic Versioning argument/gripefest! This time it was kicked off by Jeremy Ashkenas’s post why he believes Semantic Versioning is wishful thinking. Olivier Lacan chipped in further thoughts on the importance of a changelog.

Yes, Semantic Versioning is wishful thinking. Change cannot be compressed into three version numbers to guarantee safe upgrades. Developers get things wrong and forget changes such that versioning often isn’t correct, even if they wanted to follow SemVer exactly1. I thought I would write down my own versioning policies as another example for people to consider.

Continue reading


Building Systems and The Cloud

If you are building a system to run in the cloud, be prepared to spend much of your time building a resilient system.

Not a fast system. Not a very efficient system. Not a system full of fun, quirky features that users love. A resilient system because you will see performance and network issues at every connection point in your system. I hope that’s what you want.
Continue reading

Use runit!

I’ve been exploring a few new (to me!) technologies recently and runit is one that I’ve come away really impressed with. Linux distros have a few competing init services available: Upstart, systemd, runit or creaky old sysvinit. Having researched all of them and having built lots of server-side systems over the last two decades, I can firmly recommend runit if you want a server-focused, reliable init system based on the traditional Unix philosophy.

Continue reading


My Next Chapter

After 2.5 years I’ve decided to move on from The Clymb. I’m incredibly proud of our accomplishments during my time there: the site has dramatically increased in stability and scalability, the GitHub development workflow really improved code quality and we increased the size of the development team from 3 to 15. On the technical side, we moved from manually-configured cloud-based servers to Chef-managed dedicated servers, switched email providers twice, moved warehouses and rewrote our inventory and fulfillment process, integrated our logistics and fulfillment processes with an ERP system, and much more. I’m really proud to have helped grow the business while they helped me grow in management skill.

What am I doing next?
Continue reading

Setting MySQL DATETIME column defaults in Rails

Starting in MySQL 5.6.5, datetime columns can have an actual useful default of CURRENT_TIMESTAMP and MySQL will auto-populate the columns as necessary. This is incredibly handy if you ever do bulk updates in SQL, now you don’t need to remember to set updated_at! Inserting records manually will auto-populate those columns too. Let’s try it:

def up
  create_table :rows do |t|
    t.integer :value
    t.datetime :created_at, null: false, default: "CURRENT_TIMESTAMP"
    t.datetime :updated_at, null: false, default: "CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP"

Run that and we’ll see this:

ActiveRecord::StatementInvalid: Mysql2::Error: Invalid default value for 'created_at': CREATE TABLE `rows` (`id` int(11) DEFAULT NULL auto_increment PRIMARY KEY, `value` int(11) NULL, `created_at` datetime DEFAULT 'CURRENT_TIMESTAMP' NOT NULL, `updated_at` datetime DEFAULT 'CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP' NOT NULL) ENGINE=InnoDB

Notice that Rails quotes the default value, making it invalid. We can bypass this by using a custom type to define all the special logic we need and use the generic column definition method:


def up
  create_table :rows do |t|
    t.integer :value
    t.column :created_at, CREATE_TIMESTAMP
    t.column :updated_at, UPDATE_TIMESTAMP

Big Caveat: you must make sure your database’s timezone is set correctly. MySQL defaults to the system’s timezone and we set our system timezone to Pacific so everything should work fine for us.

$ mysql
mysql> select @@time_zone;
| @@time_zone |
| SYSTEM      |

Defined like that, those columns will be populated and updated any time rows are touched, not just when Rails does it.