Question 18665 in Amazon Web Sevices Question posted in
The official Amazon Web Services documentation can be found here.

Why SQS FIFO with groupID instead of using Kinesis Streams with partition key in that case? – Amazon Web Sevices

I was preparing to take the exam for AWS Solutions architecture Associate level certificate and I faced a really trick question:
Exam Question

I see Kinesis Streams as a good solution, we receive data from multiple sources and the number of cosumers are now as question said. So an streaming data with the same number of shards than the quantity of desktop would fits really well in this case.
I didn’t saw in principle SQS FIFO with groupe ID as a possible solution, it can recieve data from multiple sources and can deliver to multiple consumers in a intermitent manner using the group id, but nothing guarantee that the message will be delivered to the right desktop with SQS FIFO with GROUPID. It would depend how consumers request messages from SQS:

SQS FIFO documenation

With kinesis it would not be a problem becasue you create a shard for each consumers and thanks to partition key and the nature of the shards( records are delivery in order to consumers) it guarantee that the messages will be deliverd in order to the right consumer.SO, I can’t see why SQS FIFO with group ID is a better solution than Kinesis Streams in this case.

Could someone exaplain why SQS FIFO with groupID is a better solution than Kinesis Streams in this case?

Tags:

2

Answers


  1. 0

    The problem with the Kinesis solution is that it shards on an MD5 hash of the partition key. That means potential collisions, and consequently multiple desktop systems telemetry will be in the same shard.

    The SQS FIFO approach uses message groups IDs. If each desktop is assigned a unique message group ID then all messages from a given desktop will be in its own, unique message group and hence can be processed independently of any other desktop.

    Login or Signup to reply.
  2. 0

    My guess is that the question specifies that the data:

    "must be processed in order, independently"

    The group ID attribute in an SQS FIFO queue enforces in-order processing. If there are two messages in a queue with the same group ID, nothing can access the second message until the first message has been processed and deleted from the queue.

    Kinesis doesn’t enforce in-order, independent processing of messages within a shard. With Kinesis, two consumers could be reading from the same shard at the same time and processing that shard’s messages in parallel.

    Editing to add:

    This part of the question

    "you would like to scale the number of consumers to be possibly equal to the number of desktop systems that are being monitored."

    Indicates that you want to have possibly as many consumers as there are desktop systems, but possibly less consumers than that.

    With Kinesis you have to have a minimum of one consumer per shard. That means at a minimum the Kinesis solution would have to have the same number of consumers as desktop systems being monitored. It can’t have less consumers than desktop systems, only the same number of consumers or more.

    By contrast, SQS FIFO with Group ID has a minimum of 1 consumer, and a maximum number of consumers equal to the number of unique Group IDs. So the SQS solution is the only solution that meets the scaling requirements of the question.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search