I have a consumer that goes to Redis to persist some data
The scenario is this:
- Publishing OrderReceived
- Consumer receives the Order
- Persists this order on Redis
When the Redis Server is offline and i try to persist some data in it, it will throw an exception (RedisException) in the consumer, when this happens i want to requeue the message to maintain the sequence of the order arrival, and keep consuming the message until succeeds
2
Answers
This type of message ordering will limit consumer throughput, since you essentially have to specify a
PrefetchCount
of1
and have one single instance of your consumer running. You can use a retry policy on MassTransit to retry the error, and could use the new Kill Switch to stop the endpoint when the error limit threshold is exceeded.While you could reconfigure the error pipeline of MassTransit to NACK the message back to RabbitMQ, there is no value in it since it’s just going to be delivered right back to the consumer, but you can do it via the error pipe configuration.
You need to use the redelivery functionality of MassTransit, which is documented quite well: https://masstransit-project.com/usage/exceptions.html#redelivery
Here is the code snippet from the docs:
As you can see, you can combine message retries, which happen in-proc, with scheduled redelivery, which is, essentially, putting the message back to the same queue later in time. As it requires scheduling, you need to either use the Quartz scheduling, or the delayed messaging support from the transport (like scheduled messages in Azure Service Bus or delayed exchange in RMQ).
It is hard, however, to maintain the sequence in this case. At the same time, there is no message broker with competing consumers capability out there, which supports ordered delivery anyway. Kafka does support it, but it has a different paradigm of linear partitioned event logs and consumers only can consume from their assigned partitions, so there are no really "competing consumers" there anyway.