After talking about document-oriented databases in general in Part 1, for Part 2 I’ve written some code comparing MongDB 1.1.1, CouchDBX 0.9.1 and Tokyo Tyrant 1.4.32 in an apples to apples test.
The shootout code is on Github. I welcome patches and improvements as long as they don’t bias the tests in favor of any one system.
Results
========== Running Tokyo Tyrant tests
Using rufus-tokyo 1.0.0
user system total real
init 0.000000 0.000000 0.000000 ( 0.013781)
create 19.770000 4.260000 24.030000 ( 39.982273)
query 0.160000 0.030000 0.190000 ( 0.318070)
delete 0.000000 0.000000 0.000000 ( 0.421201)
========== Running MongoDB tests
Using mongo + mongo_ext 0.15.1
user system total real
init 0.000000 0.000000 0.000000 ( 0.005074)
create 54.710000 1.750000 56.460000 ( 57.358498)
query 0.120000 0.010000 0.130000 ( 0.155486)
delete 0.000000 0.000000 0.000000 ( 0.957453)
========== Running CouchDB tests
Using jchris-couchrest 0.23
user system total real
init 0.000000 0.000000 0.000000 ( 0.000007)
create 9.290000 0.560000 9.850000 ( 51.177824)
init is the time required to initialize the database and create any necessary indices. In practice, this number isn’t terribly relevant as this is usually an infrequent operation.
The create operation shows how long it takes for the system to bulk load 200,000 documents. You can see that Tokyo is quite fast while the Mongo client hits the CPU pretty hard. The couchrest client seems more efficient than the other two but the task still takes longer than Tokyo.
The query operation shows how long it takes to perform a non-trivial query against those 200k documents. Both Mongo and Tokyo perform about the same speed although Mongo lazy fetches the results in order to minimize network traffic when used with pagination. Tokyo returns the entire result at once AFAIK. I was not able to complete this test in a weekend using CouchDB because its view layer is so alien to me. I’d welcome help with this task.
The delete operation tests the time required to delete a subset of documents within our set of 200,000. Again, Tokyo comes out on top. Since I couldn’t perform the query in CouchDB I couldn’t delete anything either.
Conclusions? Tokyo has a reputation for being very fast and it appears to be well-founded. Couch is fast for what I could get working – I would be much more concerned about developer training and learning curve with Couch. Mongo is by no means slow but someone has to finish last. I like Mongo as an interesting mix of RDBMS and document technologies – it’s not quite as conventional as Tokyo but not as unconventional as CouchDB with its unique view layer and Erlang underpinnings. What do you think? Leave a comment and let me know!
http://github.com/actsasflinn/docdb_shootout
Using ruby-tokyotyrant 0.2.0 user system total real init 0.000000 0.000000 0.000000 ( 1.004962) create 5.240000 0.140000 5.380000 ( 7.333825) query 0.000000 0.000000 0.000000 ( 0.072963) delete 0.000000 0.000000 0.000000 ( 0.446835)Init time is actually better than the above with a clean tct file:
Using ruby-tokyotyrant 0.2.0 user system total real init 0.000000 0.000000 0.000000 ( 0.083591) create 5.180000 0.120000 5.300000 ( 7.102129) query 0.000000 0.000000 0.000000 ( 0.071614) delete 0.000000 0.000000 0.000000 ( 0.433987)Why couldn’t you try delete on CouchDB? Should be really easy if you know the ids of the documents.
Also: The view-concept is really easy. Just write a small javascript-function which emit()’s the document if it matches your needs.
I think some people could help you, if you show an example of the query you used.
Greetings,
Moritz
Just ran your tests for MongoDB and CouchDB on a Macbook Pro 2.4 GHz Core 2 Duo. The MongoDB ‘create’ tests were somewhat faster than the Couch tests on my system:
MongoDB
create (32.836103)
CouchDB
create (36.972354)
More details at http://gist.github.com/212516
Can you include some info about the system you’re using to run the tests?
Kyle, I’m using a unibody MBP 2.4Ghz with 2gb of RAM and Snow Leopard with the default Ruby.
Mike,
Some pretty significant gains can be made for MongoDB by reducing the batch size to 500, upgrading the driver to 0.16, and running a rehearsal benchmark:
MongoDB
create (26.27)
CouchDB
create (37.97)
http://gist.github.com/219053
The reason my benchmarks are running faster is the extra 1 GB of RAM on my system; MongoDB uses memory-mapped files, which can take advantage of the extra RAM.
“The reason my benchmarks are running faster is the extra 1 GB of RAM on my system; MongoDB uses memory-mapped files, which can take advantage of the extra RAM.”
You’ll probably also get a speedup if you run from Linux or Solaris.
You feed items one by one in Tokyo but it has the “putlist” command; I wonder if that would speed things up. Or did I miss something?