Firebase Question posted in
An extensive official documentation can be found here.

Adding exemptions in Firebase automatic indexing

I’m storing data in firebase with this structure:

/aggregate_data/{user id}/{product id}/data

The aggregate_data collection is the root collection ID and the rest subcollections and documents under it except data are dynamically generated. The data stored under data document is a map with a structure:

{
  'YYYYMMDD': {
     'field1': value1,
     'field2': value2,
     'field3': value3,
     'field4': value4,

  },
  .
  .
  .
  .
  .
  'YYYYMMDD': {
     'field1': value1,
     'field2': value2,
     'field3': value3,
     'field4': value4,

  },

}

The data inside this data document has now reached a point where it’s not allowing me to add (update) more data in this map since it’s being auto-indexed by Firebase and this error is produced as a result:

google.api_core.exceptions.InvalidArgument: 400 too many index entries for entity

I tried adding an exception in indexing to prevent indexing on this document so that I can add more data but not sure if I’m doing it the right way. So far I’ve tried the combinations shown in this screenshot:

enter image description here

Where sessions is one of the fields nested inside the map. Please note that the map keys YYYYMMDD are dynamically generated and contain date strings. What would be the right Collection ID & Field path combinations needed for me to stop the indexing so that I can add more data?

Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    Seemed to be any easy one since the subcollection in my case was a dynamic one I used the * wildcard as Collection ID and the field names field1, field2 etc. were known so I had to add them specifically to prevent them from being auto-indexed and this fixed the issue. One such example is given below for field name sessions: enter image description here


  2. 0

    The data stored under data document is a map with this structure.

    As far as I understand the data field is a Map that contains in terms other Map objects containing multiple fields. This looks good at first glance.

    The data inside this data document has now reached a point where it’s not allowing me to add (update) more data in this map since it’s being auto-indexed by Firebase and this error is produced as a result:

    google.api_core.exceptions.InvalidArgument: 400 too many index entries for entity

    As far as I can see in the official documentation regarding usage and limits in Firestore, the maximum number of index entries for each document is 40,000 and not 400.

    I tried adding an exception in indexing to prevent indexing on this document so that I can add more data but not sure if I’m doing it the right way.

    Even if you can manage the indexes somehow to stay below this limitation, there is another limitation that you should care about. So bear in mind that you’re also limited to how much data you can put into a document. According to the same docs:

    • The maximum size for a document: 1 MiB (1,048,576 bytes)

    As you can see, you are limited to 1 MiB total of data in a single document. When we are talking about storing text, you can store pretty much but as your documents get bigger, be careful about this limitation too.

    So storing large amounts of data into a single document might not be the best option in your cause. So I recommend you split that data into separate documents rather than nesting it into a single document. In this way, you won’t reach those limitations and you’ll be able to query the documents in any way you want.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search