Question 227182 in Amazon Web Sevices Question posted in
The official Amazon Web Services documentation can be found here.

Amazon web services – how to set interval in lambda start job

I have lambda that stops a running job and start a different glue job. It goes like this:

def lambda_handler(event, context):
    client = boto3.client('glue')
    body = json.loads(event['body'])            
    job_id_to_stop = body['job_id_to_stop']     
    job_to_start_argument = body['job_to_start_argument'] 
    client.batch_stop_job_run(                              #STOP JOB RUN
            JobName='job-to-stop',    
            JobRunIds=[
                    job_id_to_stop                 
                ]
    )
    
    client.start_job_run(                                   #START JOB RUN
            JobName = 'job-to-start',    
            Arguments = {
                '--job_to_start_argument':  job_to_start_argument                       
            }
        )
    return  {
        "statusCode": 200,
        "headers": {
            "Content-Type": "application/json"
        },
        "body": "Success"
    }

Now I know that lambda dies after running and I don’t need it to run indefinitely. However, I want to add a delay between stop_job_run and start_job_run. So after stopping the first glue job, I want lambda to wait at least 5 seconds before running the start_job_run. How do i do that?

Tags:

2

Answers


  1. 0

    Just add a time.sleep(5) in between the calls and ensure the lambda has a timeout that is large enough to submit both requests and wait for 5s, e.g. have the timeout be 10 seconds.

    Login or Signup to reply.
  2. 0

    I would use an explicit sleep() within a Lambda function with great caution: the billing model for Lambda is based on execution time, and deliberately pausing execution for N seconds would unnecessarily increase the cost.

    In this scenario, I would instead use a Step Function, orchestrating multiple Lambdas to stop/start the respective jobs (if I recall correctly you can also manage Glue jobs directly in Step Functions, but I don’t know if it fits your use case).
    This would also provide more room for customization, configuration, and reuse.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search