Use runit!

2014-07-07

I’ve been exploring a few new (to me!) technologies recently and runit is one that I’ve come away really impressed with. Linux distros have a few competing init services available: Upstart, systemd, runit or creaky old sysvinit. Having researched all of them and having built lots of server-side systems over the last two decades, I can firmly recommend runit if you want a server-focused, reliable init system based on the traditional Unix philosophy.

The point of an init system is to start and supervise processes when the machine boots up. If you’re building a modern web site, you want memcached, redis, postgresql, mysql and other daemons to start up immediately when the machine boots. Supervision means the init system will restart the process immediately if it disappears for some reason, e.g. a crash. Reliability of the init system is paramount so simplicity is a key attribute.

Unfortunately newer init systems like systemd have become increasingly complex to handle more desktop-focused requirements. Here’s a list of systemd APIs: does having 100s of public API functions and commands inspire confidence in its reliability?

The beauty of runit is its brevity and simplicity: the runsv command, which manages each process, is only 600 lines of code. In this post I want to explain how runit works and why I’m so impressed. I’m going to use the example of setting up memcached as a managed process in this post.

Creating a Managed Process

The core of runit is the /etc/sv directory. This directory contains a subdirectory for each process that runit can manage. We’ll create /etc/sv/memcached for our new memcached process. Within the process directory, you need to create an executable script called run which starts the process:

#!/bin/sh
exec /usr/local/bin/memcached -m 64

One line, dead simple. Compare that with a typical init.d script which might be 50+ lines of bash! Now that we’ve defined the process and told runit how to start it, we need to officially activate it so runit will manage it. We do this by linking the process directory to /etc/service with ln -s /etc/sv/memcached /etc/service/memcached.

The /etc/service directory contains a series of soft links representing the processes currently managed by runit. The distinction is important: /etc/sv contains process definitions, /etc/service contains actively managed processes. Those readers with a little bit of Linux administration experience will recognize this pattern: it’s exactly how the /etc/init.d and /etc/rcN.d directories in sysvinit work.

Within five seconds, runit will notice the new link in /etc/service and start the memcached process.

Controlling your Process

Once up, we can control the process via the sv command.

# sends TERM, will not restart it
sv down memcached

sv also allows us to send the slew of miscellaneous Unix signals, e.g. HUP, USR1, USR2, etc:

sv hup memcached
sv 1 memcached
sv 2 memcached

There are sysvinit compatible verbs for processes that use typical signals:

# same as down, but waits up to 7 seconds for the process to exit
sv stop memcached

Logging Process Output

Under runit, process logging becomes dead simple: your process should log to stdout only. runit provides the svlogd log helper to collect your process’s standard output and save it in a system-standard location, automatically rotating the log output as necessary. To enable automatic log management, you need to create a log directory with a run script which tells runit how to start svlogd:

#!/bin/sh
exec svlogd -tt /var/log/memcached

The /var/log/memcached directory will hold the set of log files. Within this directory you may add a config file which configures log rotation, network logging to syslog, etc.

Customizing Startup

The final piece of starting a process is controlling its environment: which env vars it sees, the user it runs as, etc. With runit, all of this is handled with the chpst (change process state) helper. If you want to run the memcached daemon as nobody, it’s a slight tweak to the run script:

#!/bin/sh
exec chpst -u nobody /usr/local/bin/memcached -m 64

So easy!

Conclusion

runit does most of the heavily lifting required for reliable, predictable daemons: starting a daemon is frequently just a single line of shell, controlling them via signals is dead simple and using svlogd completely removes any need for log maintenance or custom logrotate scripts. Each daemon no longer needs to have its own code for forking, logging, switching users, etc. By embracing the simplicity of runit, your daemons become simpler too.

If you want more info, see the runit home page or this tutorial written a few years ago.