Question 187077 in Amazon Web Sevices Question posted in
The official Amazon Web Services documentation can be found here.

Amazon web services – How to resume a step function with aws batch

I have a step function which has 2 states, lets suppose state1 and state2, state1 triggers a lambda and returns a wait state to state1, step function waits.
Lambda triggers aws batch, on completion of aws batch, it should resume the step function.
And step function should go to the next state i.e. state2.

So How to resume step function on aws batch job completion.
It’s a serverless architecture.

I have tried cloudwatch event.
Is there any alternate way?

Note: AWS Batch takes more than 20 minutes for execution.

Tags:

2

Answers


  1. 0

    Note: This will only work if your batches in aws lambda take less than 15 minutes.

    Enable wait for callback option in first lambda invoke of step function.

    enter image description here

    you can add payload something like below to use pass token, as per your input data.

    {
  "token.$": "$$.Task.Token",
  "input.$": "$",
  "callback": "true"
}

    Finally in lambda return task token on completion of aws batch by using send task token success.

    client = boto3.client('stepfunctions')
client.send_task_success(
     taskToken=event['token'],
     output=json.dumps({"key" : "value"})
)
    Login or Signup to reply.
  2. 0

    I suspect you want to use the AWS Step Functions Optimized Service Integration for AWS Batch. This integration supports the .sync / Run a Job Integration Pattern where Step Functions will manage monitoring your Batch Job and continuing the workflow execution when it completes.

    You can do drag and drop this into your workflow definition using Workflow Studio, as shown in this screen shot.

    enter image description here

    The Amazon States Language for this will follow this form:

    "Manage Batch task": {
  "Type": "Task",
  "Resource": "arn:aws:states:::batch:submitJob.sync",
  "Parameters": {
    "JobDefinition": "arn:aws:batch:us-east-2:123456789012:job-definition/testJobDefinition",
    "JobName": "testJob",
    "JobQueue": "arn:aws:batch:us-east-2:123456789012:job-queue/testQueue"
  },
  "Next": "NEXT_STATE"
}
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search