Ask Me Anything

Curious about how something works in Mongo? How to play the saxophone? How much beer costs in Manhattan? Comment below and I’ll let you know. Advice for geeks, by geeks.

If you have a question about using MongoDB, please ask on the mailing list, not here.

Please let me know what thought of this post, anonymously here or in the comments below:
Funny (0) Useful (0) Too short (0) Too long (0)
  • Luke Stanley

    Hey there, cool blog, very detailed and funny. Was wondering if you would recommend Sleepy Mongoose in order to have a COMET / Socket.io protocol synchronised JSON database, for the purposes of prototyping a realtime, ‘multiplayer’ app up to a few hundred initial users an an uncertain agile minded feature set, or if you think I should take a different route? I started working on one of my own based on some Python server code, Javascript client AND server code, including a lib called JsonSync.

    Maybe I should be using a more simple publish/ subscribe based thing.

    Thanks for your thoughts, curious for any pointers.

  • Anonymous

    Thanks!

    I’m not sure if using using sleepy.mongoose would be the best idea in your case.  Although usable, sleepy.mongoose is sort of a “last ditch” driver for people using a language without any sort of native driver or socket support.  If you’re using Python, use Pymongo and parse JSON with the the json package (that’s what sleepy.mongoose does).

  • Luke Stanley

    Thanks Kristina.

    Hmm I’m only using Python to generate Javascript and markup from Pythonic code, and a bit of server stuff – model manipulation, in this prototyping stage is done on a live distributed JSON object which is synchronised across browsers, manipulated client / browser-side. The Views are largely client-side, and would be just using a JSON object in the browser environment.
    Does that change what you’d recommend?
    Thanks, again. :)

  • Anonymous

    Are you using node.js?  If so, use that driver.  If you really want to use sleepy.mongoose, go ahead, but I wouldn’t recommend it above any of the official drivers.  Hope that helps!

  • Anonymous

    Hi,

    Let me try ask something here. :) Why do people don’t actively respond to the pull requests on mongodb project on github?

  • Anonymous

    If you look at the closed pull requests, there are 56 open requests to 77 closed requests, so we do try to merge them in.  Eliot tends to do them in batches every couple of weeks.  If you want to “expedite” your pull request, contact a kernel engineer directly that works on the code related to your pull request (do a “git blame” on the surrounding code to see who’s a good target).  They can probably do one of the following:

    * Merge it in, if it’s fine but we just haven’t gotten to it yet.
    * Let you know if it needs work: fixing whitespace, adding tests, remembering some edge case.
    * If it seems totally unworkable: it breaks backwards compatibility, uses magic numbers, is a direction we don’t want to go in (“I added transactions!”).

    Unfortunately, the developers are also super-busy, so they may be hard to get in touch with, too.  So, your best bet is to be persistent but patient.  We are insanely busy right now, which is mixed: MongoDB is exploding (yay!) but I have ~30 critical issues I’m procrastinating on by writing this comment.

  • Anonymous

    Thanks for your reply, especially in your busy hours. I find mongodb is quite interesting and want to contribute to it, but sometimes I got into dilemma:

    - some trivial fixes. people are busy and don’t care about it.
    - some harder problems. Not sure which solution would be accepted. I want to avoid the situation “hey, you’re doing it in a wrong way.”.

    As an outsider, I’ll take your advice and be patient and persistent. :)

  • Anonymous

    Definitely discuss harder problems with developers before devoting a lot of time to it.  If you get a couple of really nicely done trivial patches in, we’re more likely to merge in a bigger patch.  Small patches, well-tested patches, and platform-specific patches have been historically more likely to be applied.

  • Anonymous

    Hi, I’m loving your book “MongoDB the definitive guide”, it’s really amazing!
    But, i’m having some doubts about find and order sub-embed documents. 
    Try to imagine this structure:User embeds_many Lists and List embeds_many Items. So, how could i get all items of a specific user, ordering by some attributes of items?Could you help me? Thanks!

  • Anonymous

    Thank you, glad you’re enjoying TDG!

    Sorting sub-documents is not possible right now in MongoDB (v2.0).  If you want to sort, you’ll have to break these subdocuments out into their own documents, e.g., you could do something like:

    User: {_id : 123}
    List: {_id : 456}
    Item: {_id : 789, user : 123, list : 456, attribute : …}

    Sorting by subdocument should be possible in 2.2 with the new aggregation framework.  Keep your eye on https://jira.mongodb.org/browse/SERVER-447 if you’re interested.

  • Anonymous

    Thanks Kristina, i understood, i’ll try to do how you said.

    Thanks!!!!

  • Mhobeych

    I am having problem setting up replication with MongoDB. I am running in the AWS cloud and replicating between two regions: US East and West. Since the instance’s IP’s are Nated (Internal IP is different from External IP) MogoDB is complaining when trying to add a node. The error message I am getting is the following:

    “assertion”
    : “can’t find self in new replset config”,

    “assertionCode” : 13433,

    “errmsg” : “db assertion failure”,

    “ok” : 0I am able to use mongo –host to access the remote instance, but replication would not work. Any Idea? Thanks

  • Anonymous

    This is probably a better question for the mailing list: groups.google.com/group/mongodb-user.

  • http://twitter.com/marazmo Nikita

    Hi Kristina!
    in my podcast http://programmer.rpod.ru/ (in Russian) we often talk about programming and mongodb. If you have time – we want to invite you to participate on it. So even you don’t want to participate – can you sand me (nike-17@ya.ru) your post address – we have a little present for you.

  • Anonymous

    Thanks, I’ll shoot you guys an email!  Sounds cool!

  • http://twitter.com/marazmo Nikita

    where is an email??

  • Anonymous

    I sent you an email yesterday and just resent to the address you put in your comment.  You can also probably derive my email from my name being “kristina” and me working at 10gen.com.

  • http://www.mapmyfitness.com Chris Merz

    Greetings Kristina!  First off, let me THANK YOU for all the great work you’ve done around MongoDB.  Here at MapMyFitness.com, we’re utilizing mongo to serve up a static document store with lightweight php rest api fronting it.  We’re using v1.2.6 of the php driver, but we’re seeing unbalanced queries across the 3 replica set members.   Can you think of any reason we’d see a distribution like this (using slaveok)?

         store01.sh	    *4    156     *0     *1       0   353|0       0  1708g  1711g  18.3g     55      2.5          0       0|0     7|0    31k     2m    48 isds  SEC   14:40:56
         store02.sh	     0      0      1      1       4     116       0  1674g  1676g  19.9g      0        0          0       0|0     2|0    75k   159k    60 isds    M   14:40:56
         store03.sh	    *1     12     *0     *1       0   122|0       0  1704g  1706g  17.7g     12        0          0       0|0     0|0     7k  
  • Anonymous

    Your welcome!  Please post this on the mailing list (http://groups.google.com/group/mongodb-user), we try not to give “private” support through less Google-able sources.

  • Curiouslearn

    Hi Kristina,

    You are probably the person who knows this best, or knows people who know this. Would you say that MongoDB is a good database to use for data generated using computer simulations (discrete event simulations, agent-based models etc.). Essentially, what I have are computer models with about 30 parameters. I change the parameter values, run the computer model and then collect data on certain variables. I need to do this for many many parameter combinations and then later analyze the data. Would mongodb be good for this.

  • Anonymous

    I am not too familiar with data crunching for computer models, but it sounds similar to something someone did a while ago for a chemistry search engine: http://chemeo.com/doc/technology (see, in particular, the Indexing for High Speed Search section).  They needed to be able to search for any of dozens to hundreds of different properties for chemicals.  Hopefully that’ll help!

  • Agrapa

    Hi Kristina,

    Thanks for your blog, very informative and well written.
    I’m trying to send a query to Mongo that involves four conditions, lets call them A,B,C and D. I need the documents in the collection where A or B are true, and C or D are true. In other words: (A or B) and (C or D). I can do it from the command line like this: {$or:[{A},{B}],$or:[{C},{D}]}but I’m having trouble expressing this in PHP because of the use of arrays to represent queries. I obviously cannot do this:array(‘$or’=>array(A,B),’$or’=>array(C,D)) because you cannot have 2 separate values in an array with the same key.I could do this: array(‘$and’=>array(array(‘$or’=>array(A,B)),array(‘$or’=>array(C,D)))which amounts to a command line like this:{$and:[{$or:[{A},{B}]},{$or:[{C},{D}]}]} 

    but unfortunately Mongo does not yet know how to use an index in this case.Is there any other solution?

    Arie Grapa

  • Julien Thebault

    Hi Kristina,

    Thanks a lot for your blog, it’s very funny and instructive.
    How i can perfom map/reduce on slave ? Use “db.getMongo().setSlaveOk()” before map/reduce are best way?

    Thanks.
    King Regards.

  • Anonymous

    Thanks! Please post this on the mailing list (http://groups.google.com/group/mongodb-user), we try not to give “private” support through less Google-able sources.

  • Anonymous

    Thanks! Please post this on the mailing list (http://groups.google.com/group/mongodb-user), we try not to give “private” support through less Google-able sources.

  • DerickQ

    Hi Kristina! Quick Q on 50 Tips: I think the figures for normalized vs denormalized in (1-1 and 1-2) are switched? :-) At least in the Safari BOL Edition…

  • Anonymous

    Yes, unfortunately you are correct :-/

    (See http://oreilly.com/catalog/errata.csp?isbn=0636920019893)

  • http://twitter.com/Kerbeross Padajuan

    Hi Kristina! Im using your 4 nodes model with 2 arbiters, but i just following a log with this entry:

    Mon Jan 16 14:59:03 [conn17590] end connection 10.25.144.42:47006Mon Jan 16 14:59:05 [mongosMain] connection accepted from 10.25.144.43:53882 #17591Mon Jan 16 14:59:05 [conn17591] end connection 10.25.144.43:53882Mon Jan 16 14:59:05 [mongosMain] connection accepted from 10.25.144.42:47008 #17592Mon Jan 16 14:59:05 [conn17592] end connection 10.25.144.42:47008Mon Jan 16 14:59:06 [mongosMain] connection accepted from 10.25.144.43:53884 #17593Mon Jan 16 14:59:06 [conn17593] end connection 10.25.144.43:53884Mon Jan 16 14:59:06 [mongosMain] connection accepted from 10.25.144.42:47010 #17594Mon Jan 16 14:59:06 [conn17594] end connection 10.25.144.42:47010Mon Jan 16 14:59:08 [mongosMain] connection accepted from 10.25.144.43:53886 #17595Mon Jan 16 14:59:08 [conn17595] end connection 10.25.144.43:53886Mon Jan 16 14:59:08 [mongosMain] connection accepted from 10.25.144.42:47012 #17596Mon Jan 16 14:59:08 [conn17596] end connection 10.25.144.42:47012Mon Jan 16 14:59:09 [mongosMain] connection accepted from 10.25.144.43:53888 #17597Mon Jan 16 14:59:09 [conn17597] end connection 10.25.144.43:53888Mon Jan 16 14:59:09 [mongosMain] connection accepted from 10.25.144.42:47014 #17598Mon Jan 16 14:59:09 [conn17598] end connection 10.25.144.42:47014

    I dont know if this are a configuratión error, or is normal. Im using 1.8.3 version of MongoDB and i have Mongos process in application servers.
    Can you tell me if this is normal?¿

    Thanks for your response, and also for post of “Return of the Mongo Mailbag” are so interesting ;)

  • http://twitter.com/Kerbeross Padajuan

    Hi Kristina! Im using your 4 nodes model with 2 arbiters, but i just following a log with this entry:

    Mon Jan 16 14:59:03 [conn17590] end connection 10.25.144.42:47006Mon Jan 16 14:59:05 [mongosMain] connection accepted from 10.25.144.43:53882 #17591Mon Jan 16 14:59:05 [conn17591] end connection 10.25.144.43:53882Mon Jan 16 14:59:05 [mongosMain] connection accepted from 10.25.144.42:47008 #17592Mon Jan 16 14:59:05 [conn17592] end connection 10.25.144.42:47008Mon Jan 16 14:59:06 [mongosMain] connection accepted from 10.25.144.43:53884 #17593Mon Jan 16 14:59:06 [conn17593] end connection 10.25.144.43:53884Mon Jan 16 14:59:06 [mongosMain] connection accepted from 10.25.144.42:47010 #17594Mon Jan 16 14:59:06 [conn17594] end connection 10.25.144.42:47010Mon Jan 16 14:59:08 [mongosMain] connection accepted from 10.25.144.43:53886 #17595Mon Jan 16 14:59:08 [conn17595] end connection 10.25.144.43:53886Mon Jan 16 14:59:08 [mongosMain] connection accepted from 10.25.144.42:47012 #17596Mon Jan 16 14:59:08 [conn17596] end connection 10.25.144.42:47012Mon Jan 16 14:59:09 [mongosMain] connection accepted from 10.25.144.43:53888 #17597Mon Jan 16 14:59:09 [conn17597] end connection 10.25.144.43:53888Mon Jan 16 14:59:09 [mongosMain] connection accepted from 10.25.144.42:47014 #17598Mon Jan 16 14:59:09 [conn17598] end connection 10.25.144.42:47014

    I dont know if this are a configuratión error, or is normal. Im using 1.8.3 version of MongoDB and i have Mongos process in application servers.
    Can you tell me if this is normal?¿

    Thanks for your response, and also for post of “Return of the Mongo Mailbag” are so interesting ;)

  • http://twitter.com/Kerbeross Padajuan

    Hi Kristina! Im using your 4 nodes model with 2 arbiters, but i just following a log with this entry:

    Mon Jan 16 14:59:03 [conn17590] end connection 10.25.144.42:47006Mon Jan 16 14:59:05 [mongosMain] connection accepted from 10.25.144.43:53882 #17591Mon Jan 16 14:59:05 [conn17591] end connection 10.25.144.43:53882Mon Jan 16 14:59:05 [mongosMain] connection accepted from 10.25.144.42:47008 #17592Mon Jan 16 14:59:05 [conn17592] end connection 10.25.144.42:47008Mon Jan 16 14:59:06 [mongosMain] connection accepted from 10.25.144.43:53884 #17593Mon Jan 16 14:59:06 [conn17593] end connection 10.25.144.43:53884Mon Jan 16 14:59:06 [mongosMain] connection accepted from 10.25.144.42:47010 #17594Mon Jan 16 14:59:06 [conn17594] end connection 10.25.144.42:47010Mon Jan 16 14:59:08 [mongosMain] connection accepted from 10.25.144.43:53886 #17595Mon Jan 16 14:59:08 [conn17595] end connection 10.25.144.43:53886Mon Jan 16 14:59:08 [mongosMain] connection accepted from 10.25.144.42:47012 #17596Mon Jan 16 14:59:08 [conn17596] end connection 10.25.144.42:47012Mon Jan 16 14:59:09 [mongosMain] connection accepted from 10.25.144.43:53888 #17597Mon Jan 16 14:59:09 [conn17597] end connection 10.25.144.43:53888Mon Jan 16 14:59:09 [mongosMain] connection accepted from 10.25.144.42:47014 #17598Mon Jan 16 14:59:09 [conn17598] end connection 10.25.144.42:47014

    I dont know if this are a configuratión error, or is normal. Im using 1.8.3 version of MongoDB and i have Mongos process in application servers.
    Can you tell me if this is normal?¿

    Thanks for your response, and also for post of “Return of the Mongo Mailbag” are so interesting ;)

  • Anonymous

    Thanks! Please post your question on the mailing list (http://groups.google.com/group…, we try not to give “private” support through less Google-able sources.

  • Egenesky

    Hey Kristina,
    I just got linked to your MongoDB chess hacking article via the NoSQL Weekly newsletter.  I think the content of a lot of your blog posts would be appreciated by our community of developers at DZone.com, but I didn’t want to go reposting without your consent.  If you’re interested in getting your name/content out, shoot me an email.

    Best,

    Eric Genesky (egenesky@dzone.com)
    Content Curator
    DZone, Inc.

  • Anonymous

    Sounds good to me, sent you an email!

  • http://www.facebook.com/profile.php?id=646221069 Vaibhav Kaushal

    Hi Kristina,

    You probably work on the PHP driver for MongoDB. You could help me a little (and MongoDB for that matter). And no, I am not asking for “private support”. I am currently using PostgreSQL for creating a small website of my own based on a PHP development framework named QCubed ( http://qcu.be/ ). The framework relies heavily on something called ‘code generation’. Speaking briefly about it, code generation would read in all the catalog info and create an ORM layer in PHP for the databases. A few examples: http://examples.qcu.be would help understand. 

    Now, I am keen on MongoDB and want to use it later on. Have a 1.8 installation running on my machine too (although I dont get enough time to play around). Since MongoDB comes the closest to SQL while remaining in the NoSQL world, I thought about creating a virtual mapper for QCubed which could take in the Schema of a RDBMS and apply it to the collections on Mongo. Of course the driver can be used to add any additional data in any structure within the documents. Doing the RDBMS to MongoDB mapping has a little benefits of its own:

    1. We are effectively using the transactional ability of an RDBMS with the speeds and flexibility of Mongo. So in case some data gets irrecoverable with Mongo, the RDBMS is always there. Plus, RDBMS will handle all the writes ONLY, making Mongo do all the tough job. This will effectively use RDBMS for transactional abilities and MongoDB for all the speed. Also, mongo could be used for gathering and analyzing statistical data and logs in realtime. 

    2. With MySQL on a Blackhole engine, only MongoDB will work while schema changes will just require the schema change in MySQL. Blackhole engine will destroy incoming data so no extra space consumption (if so be the need). 

    3. IIRC, mongoDB will create a new Key or Collection if I happen to mis-spell any. With QCubed, all tables are accessed via ORM Objects. Spelling mistakes will be next to dead (unless someone issued a custom query). 

    4. I plan to make it a write twice, read once model – wherein the data is saved to both the RDBMS as well as Mongo and the read is preferably done only via Mongo and fallbacks reads are done from the RDBMS (such as when reading from secondaries to whom the changes have not propagated yet, but we know that the data was inserted). 

    All of this is going to require a lot of insight, specially in the querying mechanism. With recent development, I think it would be easier but a help is surely appreciated from you / the MongoDB team. 

    Regards

  • Anonymous

    I don’t think that adding transactional abilities or MySQL backing are very high priorities for us right now, but you might want to go on the mailing list and see what people think.  We’re happy to offer free support if you have any questions.

  • Addie Sze

    Hi Kristina,

    Trying to use mongoDump’s query ability to create incremental database dumps and hit a roadblock.

    A query in the form of
    >db.oplog.rs.find({ts:{‘$gte’:Timestamp(ISODate(“2012-01-20 20:15:27″),9899)}})
    works perfectly in a mongo shell, however won’t work in mongoDump.
    >mongodump –port 8011 -o tran –db local –collection oplog.rs -q ‘{ts:{‘$gte’:Timestamp(ISODate(“2012-01-20 20:15:27″),9899)}}’connected to: 127.0.0.1:8011Fri Feb  3 17:08:59 Assertion: 10340:Failure parsing JSON string near: ts:{:Times0x5817f2 0x5282f4 0x52866e 0xaa151b 0xa97e6d 0xa9be02 0x3d44e1d994 0x4fdfd9 mongodump(_ZN5mongo11msgassertedEiPKc+0×112) [0x5817f2] mongodump(_ZN5mongo8fromjsonEPKcPi+0×444) [0x5282f4] mongodump(_ZN5mongo8fromjsonERKSs+0xe) [0x52866e] mongodump(_ZN4Dump3runEv+0x33b) [0xaa151b] mongodump(_ZN5mongo4Tool4mainEiPPc+0x169d) [0xa97e6d] mongodump(main+0×32) [0xa9be02] /lib64/libc.so.6(__libc_start_main+0xf4) [0x3d44e1d994] mongodump(__gxx_personality_v0+0x3c9) [0x4fdfd9]assertion: 10340 Failure parsing JSON string near: ts:{:Times
    This is version 2.0.2.

    Is this a bug? Can you suggest a work around?

    Also, I realize that mongoDump version 2.1 added $timestamp capability for pulling partial oplog but I couldn’t find any documentation of its usage and syntax. Can you shed some light on this?

    You help is truely appreicated.

    Regards,
    Addie

  • http://www.facebook.com/profile.php?id=646221069 Vaibhav Kaushal

    Ok. I will. But I am not saying that MongoDB is to be backedup with MySQL (there is no need either). It is just that I want a feature of be available in a PHP framework. This helps the ‘SQL’ guys get onto Mongo very easily – that is my intention coz if I had to made the transition alone, given the way Mongo is made, I won’t have a trouble! 

    ANyways, thanks for the suggestion. Will look into it :)

  • Milko

    Have you ever thought of adding support in MongoDB for defining and traversing graph structures? I know it can be done at the client side, but it is not efficient enough. I was thinking of somethiong in the lines of the map and reduce usage of javascript, where you would get an iterator and could do the traversal on the server directly.

    Or, do you know of some good graph implementation using Mongo?

  • Anonymous

    I don’t think that has been or will be a priority for the foreseeable future.  You might want to use a graph database for that kind of thing, that’s what they’re made for.

  • Milko

    It’s a pity, because one usually needs both good document management such as with MongoDB and at the same time graph database features, it would be nice to have one database engine for that. But I guess one cannot have everything…