Voldemort

Project Voldemort

Voldemort can be described as big, distributed and fault tolerant hashtable. It promises high horizontal scalability and higher availability in the cost of lesser convenience in using it due to its primitive API.

Voldemort consists of several layers and each layer in Voldemort implements a simple storage interface that does put, get, and delete. Generally, each layer is focused in performing one function such as tcp/ip network communication, serialization, version reconciliation, inter-node routing. Voldemort keeps each of these layers separate, that means they can be configured (by mixing and matching) at run time to meet different needs.

In Voldemort, anyone can derive location of a value by knowing its key, and lookup is performed in peer-to-peer fashion. Consistent hashing is used to compute location of each key in the cluster and to prevent data shift between servers when server is added or removed from the cluster.

Voldemort supports pluggable serialization. We can choose among these kinds of serialization procedure: json, string, java-serialization, protobuf, thrift, avro-generic/avro-specific/avro-reflective, and identity.

Voldemort tolerates the possibility of inconsistency, and resolve inconsistencies at read time (versioning and read-repair). Versioning scheme in Voldemort is using vector clock to guarantee partial order over values.

Advantages

Integrity of data is preserved using versioning and without compromising system availability. Data replication, data partition and server failure are handled transparently. Voldemort layer is built based on plug-in-architecture, and we can configure these layers to suit application needs.

Disadvantages

No data found for its performance analysis, but in the other hand, LinkedIn is using it so more-or-less it has been proven in production environment. Only support key-value store, that means if we want to improve the usability (such as a file system), we need to add additional layer on top of Voldemort to perform the job

Volunteer Computing Suitability

We will need to fine tune the module in each layer to support volunteer computing which initially can be complicated, but once it is fine-tuned, it will be suitable for volunteer computing.

decentralized_storage_systems/voldemort.txt · Last modified: 2012/04/23 00:56 by julia
 
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 3.0 Unported
Recent changes RSS feed Donate Powered by PHP Valid XHTML 1.0 Valid CSS Driven by DokuWiki