I have a dispatchable job to clear and rebuild the cache for a particular model.
The cache rebuilding takes about 10 seconds.
The job can – at times – get fired multiple times sequentially and I often end up with multiple queued jobs all doing the same cache clearing and rebuilding for the same model.
I’m looking for a way of deleting any of the same job (cache rebuilding) for that particular model just before dispatching the current job.
3
Answers
Try using different queues for each of the jobs (f.e: queue per model) so you can check the queue size with
Queue::size('queue_name')
before dispatching the new job.There is no easy or %100 working way to remove duplicated job. Laravel’s Redis queue driver uses sorted sets and lists for delayed/non-delayed jobs. This answer may provide some details about it. When you push the job into the queue then you can’t remove it before it is processed. (you can try, but it is hard, the link has the answers).
What you may do is creating a control mechanism to prevent triggering cache invalidating job. Let’s say before you push your job, you set a unique identifier. You may use Redis’s SET command with
EX
andNX
option.Before you push your job you execute the first command something like this;
What you are doing is, you get the
id
of your model and create a key. You set your key with expire andset if not exists
option. If the response isOK
then you are setting this key for the first time in the given time interval(15 seconds in my example). If you getnil
response it means that, the job for the given id is stilllocked
. So you won’t dispatch it. It will provide you a 15 seconds time window for each model that will prevent you to dispatch it.—
If you can’t(don’t want) prevent the job to be triggered, then you can use the same command in a different way. Whenever you are about the dispatch the job you create a unique identifier and execute
set mymodel:id:1 some-random-string-test EX 15 NX
before dispatching it. But this time you will send identifier to the job’s constructor too. At the beginning of yourhandle
method, you will use redis’sGET
method to get the value ofmymodel:id:1
and compare it with the identifier that you sent to the job’s constructor. There will be only 1 matching identifier in 15 seconds, that means only one job will be "fully" processed. The other identifiers created for the same jobs will not be updating the the value of that redis key(NX option), therefore other job’s identifier will not matched with the value ofmymodel:id:1
. That means they will be cancelled or not fully processed because they will not pass the condition.Consider using setnx command on
phpredis
connection.Note. This is different from the
--tries
option onphp artisan queue:work
command, and retries the failed job.