Kristina Chodorow's Blog
Sleepy.Mongoose: A MongoDB REST Interface
The first half of the MongoDB book is due this week, so I wrote a REST interface for Mongo (I’m a prolific procrastinator). Anyway, it’s called Sleepy.Mongoose and it’s available at http://github.com/kchodorow/sleepy.mongoose.
Installing Sleepy.Mongoose
- Install MongoDB.
- Install the Python driver:
$ easy_install pymongo
- Download Sleepy.Mongoose.
- From the mongoose directory, run:
$ python httpd.py
You’ll see something that looks like:
================================= | MongoDB REST Server | ================================= listening for connections on http://localhost:27080
Using Sleepy.Mongoose
First, we’re just going to ping Sleepy.Mongoose to make sure it’s awake. You can use curl:
$ curl 'http://localhost:27080/_hello'and it’ll send back a Star Wars quote.
To really use the interface, we need to connect to a database server. To do this, we post our database server address to the URI “/_connect” (all actions start with an underscore):
$ curl --data server=localhost:27017 'http://localhost:27080/_connect'
This connects to the database running at localhost:27017.
Now let’s insert something into a collection.
$ curl --data 'docs=[{"x":1}]' 'http://localhost:27080/foo/bar/_insert'
This will insert the document {“x” : 1} into the foo database’s bar collection. If we open up the JavaScript shell (mongo), we can see the document we just added:
> use foo > db.bar.find() { "_id" : ObjectId("4b7edc9a1d41c8137e000000"), "x" : 1 }
But why bother opening the shell when we can query with curl?
$ curl -X GET 'http://localhost:27080/foo/bar/_find' {"ok": 1, "results": [{"x": 1, "_id": {"$oid": "4b7edc9a1d41c8137e000000"}}], "id": 0}
Note that queries are GET requests, whereas the other requests up to this point have been posts (well, the _hello can be either).
A query returns three fields:
- “ok”, which will be 1 if the query succeeded, 0 otherwise
- “results” which is an array of documents from the db
- “id” which is an identifier for that particular query
In this case “id” is irrelevant as we only have one document in the collection but if we had a bunch, we could use the id to get more results (_find only returns the first 15 matching documents by default, although it’s configurable). This will probably be clearer with an example, so let’s add some more documents to see how this works:
$ curl --data 'docs=[{"x":2},{"x":3}]' 'http://localhost:27080/foo/bar/_insert' {"ok" : 1}
Now we have three documents. Let’s do a query and ask for it to return one result at a time:
$ curl -X GET 'http://localhost:27080/foo/bar/_find?batch_size=1' {"ok": 1, "results": [{"x": 1, "_id": {"$oid": "4b7edc9a1d41c8137e000000"}}], "id": 1}
The only difference between this query and the one above is the “?batch_size=1″ which means “send one document back.” Notice that the cursor id is 1 now, too (not 0). To get the next result, we can do:
$ curl -X GET 'http://localhost:27080/foo/bar/_more?id=1&batch_size=1' {"ok": 1, "results": [{"x": 2, "_id": {"$oid": "4b7ee0731d41c8137e000001"}}], "id": 1} $ curl -X GET 'http://localhost:27080/foo/bar/_more?id=1&batch_size=1' {"ok": 1, "results": [{"x": 3, "_id": {"$oid": "4b7ee0731d41c8137e000002"}}], "id": 1}
Now let’s remove a document:
$ curl --data 'criteria={"x":2}' 'http://localhost:27080/foo/bar/_remove' {"ok" : 1}
Now if we do a _find, it only returns two documents:
$ curl -X GET 'http://localhost:27080/foo/bar/_find' {"ok": 1, "results": [{"x": 1, "_id": {"$oid": "4b7edc9a1d41c8137e000000"}}, {"x": 3, "_id": {"$oid": "4b7ee0731d41c8137e000002"}}], "id": 2}
And finally, updates:
$ curl --data 'criteria={"x":1}&newobj={"$inc":{"x":1}}' 'http://localhost:27080/foo/bar/_update'
Let’s do a _find to see the updated object, this time using criteria: {“x”:2}. To put this in a URL, we need to escape the ‘{‘ and ‘}’ characters. You can do this by copy-pasting it into any javascript interpreter (Rhino, Spidermonkey, mongo, Firebug, Chome’s dev tools) as follows:
> escape('{"x":2}') %7B%22x%22%3A2%7D
And now we can use that in our URL:
$ curl -X GET 'http://localhost:27080/foo/bar/_find?criteria=%7B%22x%22%3A2%7D' {"ok": 1, "results": [{"x": 2, "_id": {"$oid": "4b7edc9a1d41c8137e000000"}}], "id": 0}
If you’re looking to go beyond the basic CRUD, there’s more documentation in the wiki.
This code is super-alpha. Comments, questions, suggestions, patches, and forks are all welcome.
Note: Sleepy.Mongoose is an offshoot of something I’m actually supposed to be working on: a JavaScript API we’re going to use to make an awesome sharding tool. Administrating your cluster will be a point-and-click interface. You’ll be able to see how everything is doing, drag n’ drop chunks, visually split collections… it’s going to be so cool.
| This entry was posted by kristina on February 22, 2010 at 3:35 pm, and is filed under MongoDB. Follow any responses to this post through RSS 2.0. You can leave a response or trackback from your own site. |
Comments are closed.


about 6 months ago
Do you think this will end up being the preferred method of exposing MongoDB data via REST? There is currently “experimental” REST support built in to the server itself and from a performance standpoint that seems like it could be a better solution.
about 6 months ago
> Do you think this will end up being the preferred method of exposing
> MongoDB data via REST?
Unless some 3rd party has a better one that they’re going to release, yes. If people actually use sleepy.monoose, it’ll become a top-level “driver” and have its own jira section, move to github.com/mongodb, etc.
The “REST support” built into the database is and always will be pretty lean. We (MongoDB devs) don’t want to compromise overall performance by making the database do too much.
Agreed that it probably isn’t the fastest thing ever, but if you’re using REST, you’re probably not too concerned with speed anyway. It is built on top of the Python driver for a variety of reasons: Python comes with a web server, it should encourage more user contributions than if it was written in something super-speedy like C, and my boss told me to. If people really need more speed eventually, it could easily be rebuilt in C.
about 4 months ago
I am looking at writing a php interface that only exposes the database over localhost and contains a few functions for user authentication that use PHP sessions. This would be used by AJAX applications. My question is about how I should handle the more advanced queries and aggregations. Currently I would have the javascript pass a JSON object in a POST request to the PHP handler.
about 4 months ago
@Matt the user list is a good place to ask questions like this.
about 4 months ago
I have been successful in the first 3 steps of installations, but I get the following error (step-4):
# python httpd.py
Traceback (most recent call last):
File “httpd.py”, line 16, in ?
from handlers import MongoHandler
File “/home/mahmoud/downloads/kchodorow-sleepy.mongoose-9440233/handlers.py”, line 23, in ?
import simplejson as json
ImportError: No module named simplejson
any help is appreciated.
To verify that I have pymongo, I did:
# python
Python 2.4.3 (#1, Jan 21 2009, 01:11:33)
[GCC 4.1.2 20071124 (Red Hat 4.1.2-42)] on linux2
Type “help”, “copyright”, “credits” or “license” for more information.
>>> import pymongo
>>> connection = pymongo.Connection(“localhost”, 27017)
Thu Apr 15 15:22:20 connection accepted from 127.0.0.1:47319 #1
>>> Thu Apr 15 15:22:20 end connection 127.0.0.1:47319
db = connection.test
>>> db.name()
pymongo/helpers.py:140: DeprecationWarning: ‘Database.name()’ has been deprecated and will be removed. Please use ‘Database.name’ instead.
DeprecationWarning)
u’test’
about 4 months ago
@Mahmoud you need json installed (easy_install json). I’ll add that to the requirements in the README.
about 4 months ago
Hello Kristina,
Sorry for bugging you, I get the following error (using centos 64 bits).
Many thanks in advance.
# easy_install json
Searching for json
Reading http://pypi.python.org/simple/json/
Couldn’t find index page for ‘json’ (maybe misspelled?)
Scanning index of all packages (this may take a while)
Reading http://pypi.python.org/simple/
No local packages or download links found for json
Best match: None
Traceback (most recent call last):
File “/usr/bin/easy_install”, line 7, in ?
sys.exit(
File “/usr/lib/python2.4/site-packages/setuptools-0.6c11-py2.4.egg/setuptools/command/easy_install.py”, line 1712, in main
with_ei_usage(lambda:
File “/usr/lib/python2.4/site-packages/setuptools-0.6c11-py2.4.egg/setuptools/command/easy_install.py”, line 1700, in with_ei_usage
return f()
File “/usr/lib/python2.4/site-packages/setuptools-0.6c11-py2.4.egg/setuptools/command/easy_install.py”, line 1716, in
distclass=DistributionWithoutHelpCommands, **kw
File “/usr/lib64/python2.4/distutils/core.py”, line 149, in setup
dist.run_commands()
File “/usr/lib64/python2.4/distutils/dist.py”, line 946, in run_commands
self.run_command(cmd)
File “/usr/lib64/python2.4/distutils/dist.py”, line 966, in run_command
cmd_obj.run()
File “/usr/lib/python2.4/site-packages/setuptools-0.6c11-py2.4.egg/setuptools/command/easy_install.py”, line 211, in run
self.easy_install(spec, not self.no_deps)
File “/usr/lib/python2.4/site-packages/setuptools-0.6c11-py2.4.egg/setuptools/command/easy_install.py”, line 434, in easy_install
self.local_index
File “/usr/lib/python2.4/site-packages/setuptools-0.6c11-py2.4.egg/setuptools/package_index.py”, line 475, in fetch_distribution
return dist.clone(location=self.download(dist.location, tmpdir))
AttributeError: ‘NoneType’ object has no attribute ‘clone’
about 4 months ago
OK, I installed using :
yum install python-json
then
python httpd.py
=================================
| MongoDB REST Server |
=================================
listening for connections on http://localhost:27080
it worked.
Thank you Kristina!
about 4 months ago
Hello Kristina,
Have you seen the following error:
1. start sleepy.mongoose
[mahmoud@localhost kchodorow-sleepy.mongoose-9440233]$ python httpd.py
=================================
| MongoDB REST Server |
=================================
listening for connections on http://localhost:27080
2. issue _hello as:
[mahmoud@localhost ~]$ curl ‘http://localhost:27080/_hello’
curl: (52) Empty reply from server
3. sleepy.mongoose server gives the following error:
—————————————-
Exception happened during processing of request from (’127.0.0.1′, 53027)
Traceback (most recent call last):
File “/usr/lib64/python2.4/SocketServer.py”, line 222, in handle_request
self.process_request(request, client_address)
File “/usr/lib64/python2.4/SocketServer.py”, line 241, in process_request
self.finish_request(request, client_address)
File “/usr/lib64/python2.4/SocketServer.py”, line 254, in finish_request
self.RequestHandlerClass(request, client_address, self)
File “/usr/lib64/python2.4/SocketServer.py”, line 521, in __init__
self.handle()
File “/usr/lib64/python2.4/BaseHTTPServer.py”, line 316, in handle
self.handle_one_request()
File “/usr/lib64/python2.4/BaseHTTPServer.py”, line 310, in handle_one_request
method()
File “httpd.py”, line 130, in do_GET
(uri, args, type) = self.process_uri(“GET”)
File “httpd.py”, line 100, in process_uri
(uri, q, args) = self.path.partition(‘?’)
AttributeError: ‘str’ object has no attribute ‘partition’
—————————————-
about 4 months ago
It looks like the “partition” method was added in Python 2.5. Can you run ‘python –version’ to check what version you’re using?
about 4 months ago
Here are the versions:
[mahmoud@localhost]$ /usr/bin/python -V
Python 2.4.3
[mahmoud@localhost]$ /usr/bin/python
Python 2.4.3 (#1, Jan 21 2009, 01:11:33)
[GCC 4.1.2 20071124 (Red Hat 4.1.2-42)] on linux2
Type “help”, “copyright”, “credits” or “license” for more information.
>>>
about 4 months ago
Yup, that would be why! If you’d like Python 2.4 support, could you create a new issue at http://github.com/kchodorow/sleepy.mongoose/issues.
about 4 months ago
Hi,
first of all I really appreciate your work! I’m doing my diploma thesis at the moment and I search for alternatives to Amazon S3. MongoDB seems to be a very good choice and sleepy.mongoose seems to do a good job with the REST part. Is there a possibility to realize authentication over this? Like a signature for the Requests that will ensure integrity and identity?
thy
about 4 months ago
Good idea, I’ve started adding https support. Keep your eye out for version 0.02 on Github. I’ll probably also do a follow-up blog post when I release it, so subscribe to my RSS feed! That way I can let people know about some of the features that aren’t covered in this intro post (security, multiple connections, etc.)
about 4 months ago
Thy very much! I’ve subscribed to the feed
about 4 months ago
Hi Kristina.
I’m trying to connect on Mongodb with your REST interface from my Cocoa projects and I’ve got some questions:
-How did you manage the credentials? With HTTP authentication?
-Why does we need to connect first to the database? How can I close this connection?
Thanks for your answer and for your work. I have subscribed to your feed
about 4 months ago
> -How did you manage the credentials? With HTTP authentication?
Right now, sleepy.mongoose doesn’t do https connections, you can only connect over http. I’m working on https for the next version (expect it out by June). For MongoDB authentication, you can use the authenticate db command (see the core docs for more info and the Sleepy.Mongoose docs on running database commands).
> -Why does we need to connect first to the database? How can I close
> this connection?
Sleepy.Mongoose doesn’t run a MongoDB server, it just connects to them. This is kind of nice, as you can connect to multiple database servers from your application (see the Connections section of the Sleepy.Mongoose readme).
about 4 months ago
Thanks for your answer.
I will make some try and maybe post it on my blog
about 2 months ago
Hi Kristina,
How can I use skip, sort and count functionalities using REST api.
about 2 months ago
See http://wiki.github.com/kchodorow/sleepy.mongoos...
about 2 months ago
How can i insert and retrieve binary data?
about 2 months ago
How can i insert and retrieve binary data?