Question 198994 in Mongodb Question posted in
The official documentation can be found here.

Removing duplicates by specific field in MongoDB 6

Help me please. Working with Mongoose, and I want to check if there are duplicates in the "ViolationID" field and delete them, except the last one created. I think, _id: { $lt: record._id } works incorrect. May be some types of _id or record._id not matched. I’m confused, because documents in collection not deleting.

Violation.find({}, { "ViolationID": 1 })
    .sort({ _id: 1 })
    .then((violations) => {
      violations.forEach(function (record) {
        Violation.deleteMany({
          _id: { $lt: record._id },
          "ViolationID": record["ViolationID"],
        });
      });
});

Below documents that cannot be deleted

{
_id": "649adc629b36c9ee95228d96",
"ViolationID": 98,
},
{
"_id": "649add653629f115a960d498",
"ViolationID": 98
}

I tried this Remove duplicate documents based on field
and other threads, these cases don’t work for me

Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    So, adding .then or .catch after deleteMany() help it to work. This behavior is not clear to me. Do you know why this is happening?

    const removeDups = (req, res) => {
  Violation.find()
    .then((violations) => {
      violations.forEach(function (doc) {
        Violation.deleteMany({
          "ViolationID": doc["ViolationID"],
          _id: { $lt: doc._id },
        })
        // .then((res) => console.log(res));
        // .catch((err) => handleError(res, err));
      });
    })

  2. 0

    I don’t see the reason why it does not work. Anyway, may approach would be this one:

    let docs = []
db.collection.aggregate([
   {
      $setWindowFields: {
         partitionBy: "$ViolationID",
         sortBy: { _id: -1 },
         output: {
            pos: { $documentNumber: {} }
         }
      }
   },
   { $match: { pos: { $gt: 1 } } },
   { $project: { _id: 1 } }
]).toArray().forEach(doc => {
   docs.push(doc._id);
   if (docs.lenght > 10000) {
      db.collection.deleteMany({ _id: { $in: docs } });
      docs = [];
   }
})
if (docs.lenght > 0)
   db.collection.deleteMany({ _id: { $in: docs } });
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search