Question 175819 in Azure Question posted in
The official documentation to get you started can be found here.

“Can I check status of Pipeline A from Pipeline B to pass into If activity?”

How to check the status of pipeline A from Pipeline B. I have to pass the status of Pipeline A into If activity in pipeline B

I have created a pipeline A which has 3 databricks notebook and on Pipeline B i have to use If activity and pass the status of pipeline A. On False condition i have to rerun the pipeline A.
Is there any way through which I can check the status of pipeline through code/activity so that I can pass that inside the if condition.

Flow of Pipeline B will look like this:
If activity :
Pipeline A executes unsuccessful:
Rerun it
Otherwise:
Add wait activity

Tags:

2

Answers


  1. 0

    enter image description here

    You can create a master pipeline for the whole requirement

    1. Create a variable and assign the default value as ‘False’
    2. Use execute pipeline activity to execute PipelineA . Set Waitoncompletion to True
    3. Connect it to Set variable activity via ‘on completion’ (green) arrow
    4. Set the status variable to ‘True’ in the set variable activity
    5. Connect it with If activity via ‘on completion’ path (blue)
    6. In If activity , check if ‘status’ variable is true
    7. In true block call pipeline B via execute pipeline activity
    8. In false block, use web activity to call the rest API to run pipeline A again

    Here is the link for rest api to run a pipeline: https://learn.microsoft.com/en-us/rest/api/datafactory/pipelines/create-run?tabs=HTTP

    Login or Signup to reply.
  2. 0
    • Since you want to execute the pipeline A (P_A) as long as it runs unsuccessfully, you can use until loop and a flag variable to achieve this requirement.
    • I have set a flag variable with value as false to begin with.

    enter image description here

    • Inside until, I have my pipeline A (P_A), if it fails, then I would use another set variable activity and set it value to false itself.
    • If the pipeline A runs successfully, then I set flag variable value to true and thus stopping the until loop.

    enter image description here

    • The following is a pipeline JSON structure you can follow as a template for your pipeline B.
    {
    "name": "P_B",
    "properties": {
        "activities": [
            {
                "name": "Set variable1",
                "type": "SetVariable",
                "dependsOn": [],
                "userProperties": [],
                "typeProperties": {
                    "variableName": "flag",
                    "value": {
                        "value": "true",
                        "type": "Expression"
                    }
                }
            },
            {
                "name": "Until1",
                "type": "Until",
                "dependsOn": [
                    {
                        "activity": "Set variable1",
                        "dependencyConditions": [
                            "Succeeded"
                        ]
                    }
                ],
                "userProperties": [],
                "typeProperties": {
                    "expression": {
                        "value": "@not(equals(variables('flag'),'false'))",
                        "type": "Expression"
                    },
                    "activities": [
                        {
                            "name": "Execute Pipeline1",
                            "type": "ExecutePipeline",
                            "dependsOn": [],
                            "userProperties": [],
                            "typeProperties": {
                                "pipeline": {
                                    "referenceName": "P_A",
                                    "type": "PipelineReference"
                                },
                                "waitOnCompletion": true
                            }
                        },
                        {
                            "name": "Set variable3",
                            "type": "SetVariable",
                            "dependsOn": [
                                {
                                    "activity": "Execute Pipeline1",
                                    "dependencyConditions": [
                                        "Succeeded"
                                    ]
                                }
                            ],
                            "userProperties": [],
                            "typeProperties": {
                                "variableName": "flag",
                                "value": "true"
                            }
                        },
                        {
                            "name": "Set variable4",
                            "type": "SetVariable",
                            "dependsOn": [
                                {
                                    "activity": "Execute Pipeline1",
                                    "dependencyConditions": [
                                        "Failed"
                                    ]
                                }
                            ],
                            "userProperties": [],
                            "typeProperties": {
                                "variableName": "flag",
                                "value": "false"
                            }
                        }
                    ],
                    "timeout": "0.12:00:00"
                }
            },
            {
                "name": "Wait1",
                "type": "Wait",
                "dependsOn": [
                    {
                        "activity": "Until1",
                        "dependencyConditions": [
                            "Succeeded"
                        ]
                    }
                ],
                "userProperties": [],
                "typeProperties": {
                    "waitTimeInSeconds": 10
                }
            }
        ],
        "variables": {
            "flag": {
                "type": "String"
            }
        },
        "annotations": []
    }
}
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search