I have two collections:
Db.collection(groups)
Db.collection(users)
The group doc stores a map of members.
Id like to fire a function which iterates over the collection, storing aggregated data from the user document (eg. Number activities last week)
How to handle this, if the collection contains many documents?
I would solve this with a promise. But i guess i run into limits of collection.get() or a timeout during the forEach loop.
Is a paginated read (.startAt()) the solution here? What about the timeout?
export const someMethod = functions.https.onRequest((req, res) => {
let stuff: any[] = [];
let db = getFirestore();
db.collection("groups").get().then(snapshot => {
snapshot.forEach(doc => {
var newelement = {
"Members.${doc.id}.activities": doc.data().activities.lenth }
//batched write for every doc
// ...
});
res.send("ok")
return "";
}).catch(reason => {
res.send(reason)
})
});
2
Answers
On Cloud Functions v1 a single function invocation can run up to 9 minutes, on gen 2 it can run up to 60 minutes. If that isn’t enough for your workload, you will have to split it out over multiple invocations – for example by having your main function create a number of tasks.
For more on this, see:
You seem to be returning and sending back a response after the forEach. This will cause the function to run in the background which can’t guarantee execution after this point:
https://cloud.google.com/functions/docs/troubleshooting
You might want to use async/await syntax here. Promises within a forEach don’t execute how you would think. Best bet would be to iterate with a for..of over the groups. However, batchWrites only allow 500 operations at a time, so you’ll need some logic to slice(0, 500) to only get 500 operations at a time.