skip to Main Content

I wanted to perform some data processing in parallel using Bull NPM and start processing each job at the given cron Time

const Queue = require("bull"),
        
     /**
     *  initialize the queue for executing cron jobs
     **/
    
     this.workQueue = new Queue(this.queueOptions.name, {
          redis: redisConfig
     });
        
        
         this.workQueue.process((job, done) => {
              done();
              this.processJob(job)
                .then(data => {
                  global.logger.info(`successfully-executed-job ${job.id}`);
                })
                .catch(error => {
                  global.logger.error(`JSON.stringify(error)}-in-executing-job-${job.id}`); 
                });
            });
        
    // here I have included Unique JobId     
        
    this.workQueue.add({}, {repeat: { cron:"5 * * * *",jobId:Date.now()});

Any suggestions to achieve the same?

2

Answers


  1. Chosen as BEST ANSWER

    The issue is resolved now if you're facing the same issue make sure that you're referring to the correct timezone.

    Cheers!!


  2. I also faced this same issue. One thing to note with respect to the above code is that a Queuescheduler instance is not initialized. Ofcourse timezone also plays a crucial role. But without a Queuescheduler instance (which has the same name as the Queue), the jobs doesnt get added into the queue. The Queuescheduler instance acts as a book keeper. Also take care about one more important parameter "limit". If you dont set the limit to 1, then the job which is scheduled at a particular time will get triggered unlimited number of times.

    For example: To run a job at german time 22:30 every day the configuration would look like:

        repeat: { 
            cron: '* 30 22 * * *',
            offset: datetime.getTimezoneOffset(),
            tz: 'Europe/Berlin',
            limit: 1
        }
    

    Reference: https://docs.bullmq.io/guide/queuescheduler In this above link, the documentation clearly mentions that the queuescheduler instance does the book keeping of the jobs.

    In this link – https://docs.bullmq.io/guide/jobs/repeatable, the documentation specifically warns us to ensure that we instantiate a Queuescheduler instance.

    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search