Question 18595 in Amazon Web Sevices Question posted in
The official Amazon Web Services documentation can be found here.

AWS step functions, pass 1st state output to nth state input? – Amazon Web Sevices

I am creating a step function for running an ETL job when ever files are uploaded into DB. The step function involves tasks like glue jobs, crawler runs lambdas etc.

I have different clients that upload data to their respective s3 buckets and each client have their own resources like crawlers etc. and I have to execute the respective resources based on each client.

So, here in order to dynamically pick up the resource I have created a lambda to give the configurations that has these resources name. I am calling this in the first step and is it possible to refer this first states output in all other states.

Sample step function model : s3_uploaded –> get_configurations –> invoke_crawler_in get_configurations –> invoke job from get_configurations.

Tags:

2

Answers


  1. 0

    You can use ResultPath to append every state output, preserving the original input.

    The output of a state can be a copy of its input, the result it
    produces (for example, output from a Task state’s Lambda function), or
    a combination of its input and result. Use ResultPath to control which
    combination of these is passed to the state output.

    Login or Signup to reply.
  2. 0

    You use ResultPath to have the output of your state combine the input along with results generated in that state. You then need to maintain that flow through your state machine definition.

    In this example, I have four states in serial where the results from 3 of the states are combined with output. While one state sets ResultPath to null which causes the results from the state to be ignored and the state to just pass on what it got as input.

    enter image description here

    {
  "Comment": "A demo of how to keep output from earlier states to use in later ones.",
  "StartAt": "State 1 - Keep",
  "States": {
    "State 1 - Keep": {
      "Type": "Pass",
      "Parameters": {
        "state_name.$": "$$.State.Name",
        "state_start.$": "$$.State.EnteredTime",
        "some_data": {
          "key1": "val1",
          "key2": "val2"
        }
      },
      "ResultPath": "$.output.state1",
      "Next": "State 2 - Keep"
    },
    "State 2 - Keep": {
      "Type": "Pass",
      "Parameters": {
        "state_name.$": "$$.State.Name",
        "state_start.$": "$$.State.EnteredTime",
        "some_data": {
          "key1": "val1",
          "key2": "val2",
          "key3": "val3"
        }
      },
      "ResultPath": "$.output.state2",
      "Next": "State 3 - Ignore"
    },
    "State 3 - Ignore": {
      "Type": "Pass",
      "Parameters": {
        "state_name.$": "$$.State.Name",
        "state_start.$": "$$.State.EnteredTime",
        "some_data": {
          "key1": "val1",
          "key2": "val2",
          "key3": "val3"
        }
      },
      "ResultPath": null,
      "Next": "State 4 - Keep"
    },
    "State 4 - Keep": {
      "Type": "Pass",
      "End": true,
      "Parameters": {
        "state_name.$": "$$.State.Name",
        "state_start.$": "$$.State.EnteredTime",
        "some_data": {
          "key1": "val1"
        }
      },
      "ResultPath": "$.output.state4"
    }
  }
}

    So if you run this, you will get the following as output:

    {
  "Comment": "Insert your JSON here",
  "output": {
    "state1": {
      "some_data": {
        "key1": "val1",
        "key2": "val2"
      },
      "state_start": "2023-03-01T00:20:45.795Z",
      "state_name": "State 1 - Keep"
    },
    "state2": {
      "some_data": {
        "key1": "val1",
        "key2": "val2",
        "key3": "val3"
      },
      "state_start": "2023-03-01T00:20:45.795Z",
      "state_name": "State 2 - Keep"
    },
    "state4": {
      "some_data": {
        "key1": "val1"
      },
      "state_start": "2023-03-01T00:20:45.795Z",
      "state_name": "State 4 - Keep"
    }
  }
}

    If you want to learn more, you can read the Step Functions docs on Input and Output Processing or check out the module in the Step Functions workshop.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search