Mike Perham

On Ruby, software and the Internet

← →

Socket Timeouts in Ruby

March 15th, 2009 · 9 Comments

One of Ruby’s weaknesses is its poor networking performance. Much of that has to do with the net/http implementation, which uses Ruby’s awful Timeout library. The issues with Timeout are well documented. SystemTimer provides a reliable alternative that also performs better.

However I started today wondering if there was a better way. Enabling timeouts has a huge performance hit on my memcache-client library and reducing the overhead would go a long way to making it perform safely and quickly. Since C programs need socket timeouts also, I figured there had to be a low-level alternative, and indeed there is: the SO_SNDTIMEO and SO_RCVTIMEO socket options. It’s a bit involved to create a proper socket with these options but possible:

    def connect_to(host, port, timeout=nil)
      addr = Socket.getaddrinfo(host, nil)
      sock = Socket.new(Socket.const_get(addr[0][0]), Socket::SOCK_STREAM, 0)
 
      if timeout
        secs = Integer(timeout)
        usecs = Integer((timeout - secs) * 1_000_000)
        optval = [secs, usecs].pack("l_2")
        sock.setsockopt Socket::SOL_SOCKET, Socket::SO_RCVTIMEO, optval
        sock.setsockopt Socket::SOL_SOCKET, Socket::SO_SNDTIMEO, optval
      end
      sock.connect(Socket.pack_sockaddr_in(port, addr[0][3]))
      sock
    end

There are a few complexities in the code:

  • We use the low-level operations, Socket.new and connect rather than just TCPSocket.new(host, port) because otherwise we can’t set the socket options before the connection is attempted; we want to ensure the connection attempt itself is timed out also.
  • We have to look up the host via DNS by hand as some systems (*cough*, OSX) can return either IPv6 or IPv4 addresses and the address family constant used in Socket.new must match the address used in the connect statement.
  • The setsockopt method takes a native C struct so we need to construct it using the Array#pack method.

Here’s the results, from worst to best:

== memcache-client 1.7.0 with Ruby 1.8.6, normal Ruby timeouts
                                     user     system      total        real
mixed:ruby:memcache-client      14.240000   7.470000  21.710000 ( 22.173267)

== memcache-client 1.7.0 with Ruby 1.8.6, SystemTimer 1.1.1
                                     user     system      total        real
mixed:ruby:memcache-client      12.400000   1.960000  14.360000 ( 14.857924)

== memcache-client 1.7.0 with Ruby 1.8.6, raw socket timeouts
                                     user     system      total        real
mixed:ruby:memcache-client       2.750000   0.620000   3.370000 (  5.841545)

== memcache-client 1.7.0 with Ruby 1.8.6, no socket timeouts
                                     user     system      total        real
mixed:ruby:memcache-client       2.760000   0.620000   3.380000 (  5.902549)

Awesome. With raw socket timeouts, there is no performance impact! SystemTimer provides an excellent replacement for Timeout if you want to guarantee a ceiling on the time spent in an arbitrary block, but if you just need timeouts for low-level socket operations, nothing beats the operating system’s native socket timeout support.

There is a caveat in the paragraph above: low-level socket operations. memcache-client uses three IO methods: read, write and gets. The first two are low-level and time out properly, but gets is built on the low-level read operation; it has to ignore the EAGAIN error in order to ensure it returns a full line of text. So we use a hybrid approach, read and write will use the raw socket timeouts and gets will use SystemTimer. It’s not quite as fast as with no/raw timeouts but it’s definitely an improvement:

== memcache-client 1.7.0 with Ruby 1.8.6, raw socket timeouts and SystemTimer
                                     user     system      total        real
mixed:ruby:memcache-client       7.490000   1.270000   8.760000 (  9.361547)

So we’ve gone from 22 sec with Timeout to 15 sec with SystemTimer to 9 sec using raw socket timeouts where possible (Github commit). For my next trick, I figure I’ll rewrite gets to use read so I can remove the need for SystemTimer and Timeout altogether.

Tags: Ruby

9 responses so far ↓

  • 1 Dmytro Shteflyuk // Mar 25, 2009 at 9:56 am

    Hey Mike,

    Thanking you for the great sharing! I was looking for some technique to handle timeouts in my Sphinx ruby client for a long time. Currently we (Scribd) wrap all calls in SystemTimer blocks, but your solution is much cleaner and it would be great improvement. Keep up your good work.

    PS. We are using your memcache-client too and really enjoyed it. You rock!

  • 2 David R // Apr 12, 2009 at 5:16 pm

    Apparently SO_RCVTIMEO/SO_SNDTIMEO are not supported in solaris, which renders the most recent version of the memcache-client pretty impotent. I am getting these errors:

    Errno::ENOPROTOOPT: Option not supported by protocol

    And some googling on SO_RCVTIMEO and solaris didn’t come up with any great solutions.

  • 3 raggi // Jun 6, 2009 at 5:12 am

    What was the benchmark, and why is there such a massive discrepancy?

    Just because timeout is broken in it’s capabilities, I can’t see why it would be over 200% slower.

  • 4 mperham // Jun 6, 2009 at 7:41 am

    raggi, the benchmark is part of the test suite in the memcache-client source at http://github.com/mperham/memcache-client

    And as an update, I’ve had to roll back any use of socket timeouts in Ruby. They don’t work on all platforms (like Solaris mentioned above).

  • 5 cheapRoc // Jun 6, 2009 at 10:37 am

    @mperham excellent, excellent article and code.

    one question though, i don’t seem to understand rolling back features just because all platforms do not support them.

    shouldn’t the features that we implement be tailored to each platform, utilizing the best that they provide, not dulling them down by spreading basic functionality across all of them…

    personally, i completely ignore Windows all together because i’m not best fit to provide that Windows support. the power of OSS compels you. ;)

  • 6 Aaron Patterson // Jun 6, 2009 at 12:42 pm

    Hi! I patched memcache-client to make it fast by using a buffered IO object and non-blocking reads. I think it will work on all platforms. Bonus is that you don’t need SystemTimer at all.

    Here are my benchmarks:

    http://gist.github.com/124957

    Here is my patch:

    http://github.com/tenderlove/memcache-client/commit/5d7f38ca066fcc9059432186d15c8fc50fcdfbcf

    Oh, I sent a pull request too. :-)

  • 7 mperham // Jun 6, 2009 at 1:10 pm

    @cheapRoc, memcache-client is designed to work on all ruby platforms. It’s included with Rails and so compatibility is the #1 priority.

    If you need absolute maximum speed, you should upgrade to fauna/memcached, which in my experience is extremely fast but difficult to use on anything but Linux.

    And, as Aaron has proven, often Ruby can be made quite fast without sacrificing compatibility if you know what you are doing. Thanks Aaron!

  • 8 Rodjer // Jun 16, 2009 at 1:37 am

    Mike, any impression on how memcache client fares against memcached?

    While the memcache gem is certainly more managable and well supported in eg. Rails and related projects, then libmemcached appears to be the C lib that gets the most love.

    Any thoughts on where we’re headed and on the pros/cons of either library?

    Thanks

  • 9 sergeych // Nov 7, 2009 at 9:07 pm

    doesn’t work on Mac OS 10.5/ruby1.9. Does anybody ever wrote a decent sockets lib interface? Maybe everybody use dl direct calls for real work? Sockets in ruby, even 1.9, looks more like a joke…

Leave a Comment