Question 107944 in Amazon Web Sevices Question posted in
The official Amazon Web Services documentation can be found here.

Amazon web services – step function to N times for a task

I am finding a way to do this in , My scenario is I want to repeat a step until I am not finished my dynamic number of files processing in step function for example

step1 -> step2 -> step2 -> step2 -> step3 -> done

step 2 will be dependent on number of files I needed as output from step 1 . like If I get 30 files I want to divide these files in 2 chunks and run step2 for 3 times not 30 times so how I can make a state machine (preferably in python ckd) . any clue even in json would be highly appreciated.

EDIT after searching I came to know we call it map task like mentioned here https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-map-state.html but not getting how I can done this using cdk .

for example if my lambda look like this :

 sfn_step_1 = tasks.LambdaInvoke(self, "step1",
                                        lambda_function=step1_lambda,
                                        output_path="$.Payload",
                                        payload=stepfunctions.TaskInput.from_object({
                                            "payload.$": "$",
                                            "execution_id.$": "$$.Execution.Id",
                                            "taskToken.$": "$$.Task.Token"
                                        })
                                        )
        sfn_step_2 = tasks.LambdaInvoke(self, "step2",
                                        lambda_function=step2_lambda,
                                        output_path="$.Payload",
                                        payload=stepfunctions.TaskInput.from_object({
                                            "payload.$": "$",
                                            "execution_id.$": "$$.Execution.Id",
                                            "taskToken.$": "$$.Task.Token"
                                        })
                                        )
        sfn_step_3 = tasks.LambdaInvoke(self, "step3",
                                        lambda_function=step3_lambda,
                                        output_path="$.Payload",
                                        payload=stepfunctions.TaskInput.from_object({
                                            "payload.$": "$",
                                            "execution_id.$": "$$.Execution.Id"
                                        })
                                        )

        state_machine_definition = sfn_step_1.next(sfn_step_2).next(sfn_step_3)

like in this I want to repeat step2 multiple times depending upon number of records I get from step2

Tags:

2

Answers


  1. 0

    I wrote here an example of implementing a map in the step functions. I hope it will be helpful.

    from aws_cdk import aws_stepfunctions
import aws_cdk.aws_stepfunctions_tasks as tasks
from aws_cdk import aws_lambda
...
...
...
        self.get_files = aws_lambda.Function(
            self,
            ...
        )
        # should retrun list of the files like below:
        # return [{"file": "file1"}, {"file": "file2"}...]
        
        self.task_2 = aws_lambda.Function(
            self,
            ...
        )

        self.get_files_task = tasks.LambdaInvoke(
            self,
            lambda_function=self.get_files,
            result_path="$.File_List",
            output_path="$",
        )
        self.map_state = aws_stepfunctions.Map(
            self,
            max_concurrency=1,
            items_path=aws_stepfunctions.JsonPath.string_at("$.File_List.Payload"),
            result_path="$.Single.File",
        )
        self.start_task_2 = tasks.LambdaInvoke(self, id="name", lambda_function=self.task_2)

        self.map_state.iterator(self.start_task_2)
        self.definition = self.get_files_task.next(self.map_state)
        self.state_machine = aws_stepfunctions.StateMachine(self, "example", definition=self.definition)
    Login or Signup to reply.
  2. 0

    What about "choice" after the step 2.

    https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_stepfunctions.Choice.html

    something like this

      .next(new sfn.Choice(this, 'Repeat step2 ?')
    // Look at the "status" field
    .when(sfn.Condition.stringEquals('$.status', 'REPEAT'), step2)
    .when(sfn.Condition.stringEquals('$.status', 'MOVE_ON'), step3)
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search