I have a multi-tenant solution where jobs are submitted from an on-premise system for background processing. Once the result is available, my design is to write a poller from on-premise system which will take the results and send it to the original requester.
My solution runs on AWS and I was planning to use SQS to store the results once available. The poller will dequeue an item from this queue and if for some reason poller goes down visibility timeout of the queue ensures that result is not lost.
However, since my solution is a multi-tenant one how can I ensure that a on-premise poller gets only the result of that tenant. If I have a single SQS then poller will get a random result of some other tenant. I know that SQS does have a concept of group ids but there is no provision for a poller to query based on a group id.
I would like to avoid creating a SQS per tenant. Is there any better solution for this? Note that my solution is AWS specific and need not be portable to other cloud vendors.
2
Answers
SQS queue per tenant is the only way.
It is not possible to selectively pull messages from an Amazon SQS queue.
The
MessageGroupIdyou mention is specific to FIFO queues and simply ensures that messages with the same Group ID are delivered in-order. It is not related to the ability to pull a specific subset of messages.You will either need separate queues per tenant, or a polling mechanism that works for all tenants.