Comments on: Cassandra and EventMachine

By: Mike Perham

Mike Perham — Fri, 14 Oct 2011 22:06:25 +0000

This is not the right venue for detailed technical help. One core issue with your impl is that you are still just doing one line at a time (because you only create a single fiber). Why bother with EM + Fibers if you are still basically single threaded?

By: Chris

Chris — Fri, 14 Oct 2011 21:17:14 +0000

Hi Mike et al.,

I’m relatively new to Cassandra and Eventmachine so I have to learn a lot My goal is really simple. I wanna read a large file (120 MB) line by line and insert every row into a Cassandra DB. Sounds simple but I’m trying it for 2 days now

#!/usr/bin/env ruby

require ‘rubygems’
require ‘cassandra’
require ‘eventmachine’
require ‘thrift_client/event_machine’

EM.run do
Fiber.new do
rm = Cassandra.new(‘RankMetrics’, “127.0.0.1:9160″, :transport => Thrift::EventMachineTransport, :transport_wrapper => nil)
rm.clear_keyspace!
file = File.open(“us_100000.txt”)
read_chunk = proc do
10.times do
if line = file.gets
rm.insert(:Domains, “#{line.downcase}”, {‘domain’ => “#{line}”})
else
EM.stop
end
end
EM.next_tick(read_chunk)
end
EM.next_tick(read_chunk)
end.resume
end

AFAIK EventMachine doesnt support asynchronous fs read/write, right? So I tried it with the chunks of 10 lines per tick but the code seems to block. I’m at my wit’s end. Do you have a hint for me whats going wrong there?

Cheerio,
Chris

By: Christian

Christian — Mon, 09 Aug 2010 19:18:07 +0000

I have been investigating using Cassandra, Sinatra, EventMachine and Fibers for the purposes of building a highly scalable and distributed API. I have fully enjoyed and learned quite a bit from your blog and presentations but I am finding it difficult to develop and Sinatra/Eventd application that pools Cassandra connections. Can you provide a more detailed example that illustrates this?

By: mperham

mperham — Sun, 14 Mar 2010 15:56:04 +0000

Blair, look at the Cassandra and thrift_client gems. Their APIs are blocking – there’s no way the gems could be used with the more traditional EM callback style. You can certainly have a callback-based Cassandra client but you would need to start from scratch and develop a new API.

By: Blair

Blair — Sun, 14 Mar 2010 07:01:46 +0000

Correct me if I am wrong, but looking at this, it seems to be a blocking implementation, that is using a separate thread to communicate with Cassandra.

Are there any Cassandra Protocols for EventMachine that can run within the Reactor thread? Is that a silly thing to expect with Cassandra (I am new to Cassandra and Eventmachine)

By: Ryan King

Ryan King — Wed, 10 Feb 2010 17:43:57 +0000

I’m glad we’re going to have this support. Perhaps you can add an example like the above to the README or a wiki page?