"post"
Tue, 02 Feb 2010 18:59:51 EST
cl-mongo
mongo is a scalable, high-performance, open source, schema-free, document-oriented database. I was introduced to mongo at the new-york mysql meetup. Two things made mongo look attractive: inter-operability and document centric storage.
I'm familiar with the elephant persistence framework in lisp. However elephant objects are not readable (as far as I know) in languages other than lisp.That makes inter-operating with other platforms difficult. A traditional rdms requires some sort of schema if you want to use it effectively. Mongo on the other hand is optimized for the the kind of free form document storage I'm looking for.
Mongo comes with set of drivers but was missing was a lisp driver. This looked like a good project to get better acquainted with lisp and mongo.
So I set out to try to write one and the result is cl-mongo. At this stage it's close to having the capabilities I'm looking for, but cl-mongo is obviously a work in progress.
Mongo stores documents in a collection in a database. Internally, as far as I can tell from the protocol, the combination of database and collection name make up a unique namespace on the server in which documents can be stored.
Each document itself is a set of key-value pairs, with the keys typed as utf8 encoded strings. The value side supports a variety of types, from the usual primitives (float, int, Boolean) up to regular expressions and code.
A document has a unique id, associated with reserved key word "_id". If this id is not client generated the server will provide one. The server supports a whole host of database commands. These are also structured as sets of key-value pairs, but such commands and their response aren't necessarily documents.
One of the main design ideas is to recognize different layers at which the communication operates:
- layer 0 : byte-level at which applications exchange data.
- layer 1 : mongo serialization protocol and mongo specific types.
- layer 2 : native code types.
- layer 3 : mongo container types and operations on them.
- layer 4 : syntactic sugar.
An example of a mongo specific type is mongo's representation of an array as a collection of key value pairs with the keys being stringified indexes like so:
{ '1' : elem1 } , { '2' : elem2 } An obvious choice for it's native code type counterpart is the list.
Since mongo documents are collections of key-value pairs an associative array like a hash-table serves quite well as a mongo container type. In addition to the hash-table there is a mongo document which is just a hash table with a unique id attached.
As far as the basic api was concerned I wanted to stay as close as possible to the db shell for the java-script client, as detailed in the mongo documentation. That way it's possible to use the commands from the mongo reference manual when using cl-mongo from the repl.
3474143991