Question 118035 in Mongodb Question posted in
The official documentation can be found here.

Speed up MongoDB Query Response | Discord Bot

I didn’t think this would be an issue, but as the database I am using continues to grow, I’ve noticed that the responses from my program are getting slower and slower (which makes sense, more stuff to look through to return).

Now my question is how would I be able to speed up the performance of my response because it takes quite some time to do the following as of right now:
I want to sort and return to the top 10 results from a query search, currently, my code looks something along the lines of:

alldocuments = self.bot.db.nameofdb.find()
alldocuments = list(map(lambda x: x["userId"], alldocuments ))
_documentCount = [(i, alldocuments.count(i)) for i in set(alldocuments)]
_documentCount.sort(key = lambda x: x[1], reverse=True)
docCount = list(filter(lambda x: ctx.guild.get_member(int(x[0])), _documentCount))
        
        
i = 0
while i < 10:
     if not ctx.guild.get_member(int(docCount[i][0])):
          i+=1
          continue
     print(docCount[i][1]})
     i += 1

Some information:

Database consists of thousands of entries as follow:

Entry Ex. 1:
{
_id: 62338b6994b3773415bd7efc
uid: "849551661"
gacha_type: 100
item_id: ""
count: "1"
time: "2021-12-25 17:42:58"
name: "Magic Guide"
lang: "en-us"
item_type: "Weapon"
rank_type: "3"
id: "1640423160000228061"
userId: 738362958253522976
}

Entry Ex. 2
{

_id: 62338b6994b3773415bd7efe
uid: "849551661"
gacha_type: 100
item_id: ""
count: "1"
time: "2021-12-25 17:42:58"
name: "Noelle"
lang: "en-us"
item_type: "Character"
rank_type: "4"
id: "1640423160000227861"
userId: 738362958253522976

}

Now my database currently exists with over 100k of these entries, and I want to sort them by userId in the order of which "userId" has the greatest amount of entries.

A sample result should be something along the lines of:

1. 25k - userId
2. 15k - userId
3. 10k - userId
4. 8k - userId
5. 5k - userId
6. 1k - userId
7. 200 - userId
8. 100 - userId
9. 50 - userId
10. 25 - userId

where anyone with under 25 entries would just be ignored.

Tags:

2

Answers


  1. 0

    Using MongoDB, you’re able to directly include those filters that you are applying into your query. This should impact your runtime by loading fewer data.

    documents = self.bot.db.nameofbd.find({"queryparam": True})
sorted_documents = documents.sort("queryparam": pymongo.DESCENDING)
    Login or Signup to reply.
  2. 0

    It’s a bit difficult to give an answer without proper information (i.e. maybe 2-3 more sample input documents and sample result). However, could be this one:

    db.collection.aggregate([
  {
    $group: {
      _id: {
        location: "$location",
        name: "$name"
      },
      count: { $sum: 1 }
    }
  },
  { $sort: { count: -1 } },
  { $limit: 10 }
])
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search