Question 116428 in Mongodb Question posted in
The official documentation can be found here.

Mongodb – Mongo not creating all docs in array and not giving errors

I have a long array of docs to create. When I create them I get no errors.

const docsJson =[some array of json of docs to create]
 
const orders = await MySchema.create(ordersJSON);
// orders.length returns the same number of docs as  docsJson

But when I search for the new docs, only some were created.


const actualOrdersCreated = await MySchema.find({ _id: { $in: orders.map((p) => p._id) } });
// actualOrdersCreated.length returns less docs than in docsJson

What’s causing this?

Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    This was due to having a ttl (time to live) index on one mongo database and not the other. I was copying docs over from a database. The index on the first database was:

    $ mongo "mongodb+srv://....database-1-url"
>> db.myschema.getIndexes()
[
        {
                "v" : 2,
                "key" : {
                        "_id" : 1
                },
                "name" : "_id_"
        },
        {
                "v" : 2,
                "key" : {
                        "paidOn" : 1
                },
                "name" : "paidOn_1",
                "background" : true
        }
]

    But the database I was working with had expireAfterSeconds.

    $ mongo "mongodb+srv://....database-2-url"
>> db.myschema.getIndexes()
[
        {
                "v" : 2,
                "key" : {
                        "_id" : 1
                },
                "name" : "_id_"
        },
        {
                "v" : 2,
                "key" : {
                        "expireAt" : 1
                },
                "name" : "expireAt_1",
                "background" : true,
                "expireAfterSeconds" : 86400
        },
        {
                "v" : 2,
                "key" : {
                        "paidOn" : 1
                },
                "name" : "paidOn_1",
                "background" : true
        }
]

    So mongo was deleting the new docs where the expireAt field had an old date.

    To fix it I ran await Order.syncIndexes(); in a script. This cleared the index to [ { "v" : 2, "key" : { "_id" : 1 }, "name" : "_id_" } ]. This is fine for my purpose. But the index isn't the same as the 1st database. The paidOn key is no longer indexes.

    What I thought helped but didn't

    At first I thought the issue was due to the large size of the jsonDocs.

    I had objects with fields that had large base64 strings for images. These were placeholders, supposed to be replaced with http urls for the images.

    After I removed the base64 strings I was able to upload the documents. I thought this helped but it was just speeding things up. It takes 1 minute for mongo to check up on expired docs.


  2. 0

    I think your data is to large.
    The maximum BSON document size is 16 megabytes.
    Reference: https://www.mongodb.com/docs/manual/reference/limits/

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search