Question: How to customize long-running job without attempting multiple time after each retry_after seconds?
I have one job which will take 1 to 3 hours to run,
I already created job-based on laravel documentation, here is my job file.
<?php
namespace AppModulesCsvJobs;
use AppJobsJob;
use IlluminateBusQueueable;
use IlluminateContractsQueueShouldQueue;
use IlluminateFoundationBusDispatchable;
use IlluminateQueueInteractsWithQueue;
use IlluminateQueueSerializesModels;
use IlluminateSupportFacadesDB;
use IlluminateSupportFacadesStorage;
use IlluminateSupportStr;
use LeagueCsvReader;
use Phone;
/**
* A single excel import job, which can be pushed on to a queue
*/
class UploadCsvDataInTable extends Job implements ShouldQueue
{
use InteractsWithQueue, SerializesModels, Dispatchable,Queueable;
public $timeout = 172800;
/**
* The excel to import
*
* @var AppBulkUpload
*/
protected $csvUpload;
/**
* Create a new job instance.
*
* @param AppFeedImport
*
* @return void
*/
public function __construct(CsvUpload $csvUpload)
{
$this->csvUpload = $csvUpload;
}
public function handle()
{
app(CsvUploadService::class)->uploadCsv($this->csvUpload);
}
}
here is Laravel document to specify a timeout for jobs.
here is code for how I am calling that job.
UploadCsvDataInTable::dispatch($csvUpload)->onConnection('redis')->onQueue('low');
my command for queue:work In supervisor.
php artisan queue:work --queue=high,low,default --sleep=3 --tries=3
here is my configuration for queue & horizon
// horizon.php
'production' => [
'supervisor-1' => [
'connection' => 'redis',
'queue' => ['high', 'default', 'low'],
'balance' => 'simple',
'processes' => 6,
'tries' => 3,
],
],
//queue.php
'redis' => [
'driver' => 'redis',
'connection' => 'default',
'queue' => env('REDIS_QUEUE', 'default'),
'retry_after' => 90,
'block_for' => null,
],
I can see because of retry_after my job is attempting multiple time and after reaching 3 try as per horizon configuration its throwing MaxAttemptsExceededException.
if I increase $timeout to 24 hours I am getting duplicate records in my db as retry_after is attempting that job multiple time.
is there any way I can set custom retry_after for this job?
2
Answers
I have created another connection for long-running jobs and its working properly for me.
created new supervisor connection in horizon.php for long running process
and new redis connection in queue.php
in database.php added new queue for long running jobs.
also don't forgot to call jobs using onConnection and onQueue to specify from which queue jobs should execute.
you don’t need to set
retry_after
, you need to set tries to 1https://laravel.com/docs/8.x/queues#max-attempts