Faktory is my new background job system which brings Sidekiq-like background jobs to all languages. Want Sidekiq in PHP? Python? JavaScript? You got it!

faktory ui

Faktory 0.9 has just been released with a major architectural overhaul. I've replaced the previous storage engine, RocksDB, with Redis. This change had hugely important, very good downstream effects!

Want to try Faktory? The wiki has everything you need!

Why Redis?

So why did I replace RocksDB? RocksDB has two advantages over Redis:

The problem is that everything else is a disadvantage:

Redis, on the other hand:

Now instead of a directory full of obscure files, we have a single faktory.rdb file with all data. Now we can use redis-cli and RDB tools to introspect the datastore, monitor commands in real-time, use existing Redis monitoring integrations to monitor Faktory's storage. When we build on the shoulders of giants, the view is pretty good.

The only issue I had to solve is how to "embed" Redis? I can't link it into the process directly so instead I updated Faktory to start Redis as a child process and maintain a named socket to it. By keeping it local, using a named socket, we ensure our usage of Redis is both high performance and secure (since Redis does not listen on the network).

faktory logo

What's Upcoming?

With 0.9 out, I'm very close to releasing Faktory Pro, my first commercially supported version of Faktory! The initial feature list:

  1. Cron jobs - enqueue jobs on a periodic schedule
  2. Expiring jobs - throw away jobs which have passed a given expiration date
  3. Redis Gateway - expose Redis on a TCP port so you can attach a replica to it for real-time replication

This feature list is very likely to grow over time.

See the Faktory wiki for how to install and lots of documentation. Join us in the chatroom or open an issue if you have questions.

comments powered by Disqus