inspector_gadget_hd_cartoon_wallpaper-wide

Inspeqtor 0.6.0 – init.d, cron and statsd!

Inspeqtor is my new take on application infrastructure monitoring. You use it to verify rules about your database, web server, caches, etc and if anything is wrong, Inspeqtor will alert you. After the initial release 3 weeks ago, I was distracted a bit by a DMCA takedown from a similar project but thankfully that went away quickly. Since then I’ve added several awesome new features!

What’s new?
Continue reading

Stupid Go tricks in Janus

I’ve used Janus as my main text editor for the last few years; it’s a good solution for people like myself that want to quickly install not just MacVim but also a curated set of plugins necessary to have a productive IDE for Ruby/JS development.

Side note: learn vim if you’re a developer and don’t know it already. Every Unix environment in the world has vim installed. Learning vim is an excellent investment of your time and ensures you can be productive everywhere, not just your own machine.

The Janus developers recently added a better Go plugin and I wanted to call out a few features that are really cool. Put these lines in your ~/.vimrc.after:

au FileType go nmap <leader>c <Plug>(go-coverage)
au FileType go nmap gd <Plug>(go-def)
au FileType go nmap <Leader>r <Plug>(go-rename)
  1. Typing “\c” will open an HTML code coverage report in your browser for the current package showing test coverage.
  2. Typing “gd” will “goto definition” of the word under the cursor, allowing you to jump to methods or types.
  3. Typing “\r” will rename the thing under the cursor. Real, type-safe renaming in vim, awesome!

Go’s static type system can be cumbersome at times but does allow automatic refactoring within editors. I’m happy to see these theoretical benefits finally trickle down to improve my daily work.

highres_358490542

Why Go?

When I decided to build Inspeqtor, I had a fundamental choice: what language should I build it in? I’ve worked in Ruby for the last 8 years so it was a natural choice: “use the tool you know best” is never a bad choice when solving your own problem.

However I’m not building something for myself: I’m building a product that will be used by thousands of others. Since Inspeqtor is an infrastructure monitoring tool, it needs to run 24/7 efficiently and reliably. I also want everyone to be able to install and use Inspeqtor with the absolute bare minimum of hassle. This means minimizing third-party dependencies like the Ruby VM itself, gems, etc. In the end, I needed to select the right tool for the job, not just the tool I know.

Ultimately these requirements led me to two options: Go or Rust. Both can build binaries which have no runtime dependencies at all. Both are reliable and fast. Rust has a huge amount of potential but version 1.0 is still not ready yet — they’re still working out syntax details — and learning their memory ownership model has a definite learning curve. I chose Go because of its maturity and the simplicity of the language.

I want to call out two features that make Go so nice to work with:

Easy concurrency

Modern languages must have a better concurrency story than “thin wrapper around the POSIX thread API”. Go’s goroutines and channels are a simple but powerful abstraction and easier to use safely than traditional threads. Here Inspeqtor gathers the current metrics in parallel for the entities it is monitoring. I’m still trying to figure out best practices for handling errors and ensuring timeouts in goroutines. Google’s Context pattern looks like a strong contender to solve that problem.

Full development workflow

To paraphrase the poet John Donne: “No programming language is an island”. Go ships with tools for testing, code profiling, documentation, cross compiling and syntax formatting. There was only one thing I felt that Go should provide that it doesn’t: an assert package for test code. Here’s some example code which uses a 3rd party assert package I like. Adding if blocks here for every check would really hurt the readability of the code.

So far I’m very pleased with how well Go has worked out for Inspeqtor. I’ve been running Inspeqtor on my server for about two months now and never once has it gone over 10MB of RAM or any significant CPU usage. Here’s Inspeqtor monitoring itself:

$ sudo inspeqtorctl status
Inspeqtor Pro 0.5.0, uptime: 54h14m14s, pid: 11645

Service: inspeqtor [Up/11645]
  cpu:system                  0.1%           
  cpu:total_system            0.0%           
  cpu:total_user              0.1%           
  cpu:user                    0.1%       90%
  memory:rss                  8.97m      10m

Every language has strengths and weaknesses. For this purpose, Go has worked out great.

PS C and C++ were never considered. I vowed in 1997 to never manage my own memory again and not for a single day have I ever regretted that decision.

Introducing Inspeqtor

I’ve written server-side applications for a decade now, and monitoring the components of your application is critical but painful. What monitors the CPU and RAM usage of your custom daemons? What monitors Redis, MySQL, memcached and the other parts of your system to ensure they are all behaving normally? What if I told you you could do all that and set it up in less than 5 minutes?

Say hello to Inspeqtor!

logo-inspeqtor

Continue reading

2013-07-21 11.00.06c

Don’t Daemonize your Daemons!

For years developers have followed the same arcane dozen steps to create a long-lived daemon process on Unix-based systems. These steps were state of the art in 2000 but they are no longer best practice today. Jake Gordon’s recent blog post on daemonizing Ruby processes is 100% factual but his approach is not recommended these days. Your application code should not be dealing with PID files, log redirection or other low-level concerns.
Continue reading

Interior_view_of_Stockholm_Public_Library

Library Versioning

It’s time for our annual Semantic Versioning argument/gripefest! This time it was kicked off by Jeremy Ashkenas’s post why he believes Semantic Versioning is wishful thinking. Olivier Lacan chipped in further thoughts on the importance of a changelog.

Yes, Semantic Versioning is wishful thinking. Change cannot be compressed into three version numbers to guarantee safe upgrades. Developers get things wrong and forget changes such that versioning often isn’t correct, even if they wanted to follow SemVer exactly1. I thought I would write down my own versioning policies as another example for people to consider.

Continue reading

cloud2_55171579

Building Systems and The Cloud

If you are building a system to run in the cloud, be prepared to spend much of your time building a resilient system.

Not a fast system. Not a very efficient system. Not a system full of fun, quirky features that users love. A resilient system because you will see performance and network issues at every connection point in your system. I hope that’s what you want.
Continue reading

Use runit!

I’ve been exploring a few new (to me!) technologies recently and runit is one that I’ve come away really impressed with. Linux distros have a few competing init services available: Upstart, systemd, runit or creaky old sysvinit. Having researched all of them and having built lots of server-side systems over the last two decades, I can firmly recommend runit if you want a server-focused, reliable init system based on the traditional Unix philosophy.

Continue reading

photo-5

My Next Chapter

After 2.5 years I’ve decided to move on from The Clymb. I’m incredibly proud of our accomplishments during my time there: the site has dramatically increased in stability and scalability, the GitHub development workflow really improved code quality and we increased the size of the development team from 3 to 15. On the technical side, we moved from manually-configured cloud-based servers to Chef-managed dedicated servers, switched email providers twice, moved warehouses and rewrote our inventory and fulfillment process, integrated our logistics and fulfillment processes with an ERP system, and much more. I’m really proud to have helped grow the business while they helped me grow in management skill.

What am I doing next?
Continue reading