Azure Question posted in
The official documentation to get you started can be found here.

Azure – How to use minimum number of web activities to send notifications in a ADF pipeline?

I am using Web Activity to send notifications of pipeline run in MS Teams. My requirement is to use minimum number of web activities to reduce the cost. At the same time I want notification in case any of the activities fail. Please suggest the ideal connection keeping in mind the above requirement.

enter image description here

I want to use as small number of activities as possible to send notification. I tried to connect using only one web activity but it fails.

Tags:

2

Answers


  1. 0

    It will be a complex task and involves lot of activities to send the message for any of the activity failure in a pipeline.

    You can use Execute pipeline activity for this requirement. But this requires another parent pipeline. Remove the web activities and put all of your required activities in a pipeline and call that pipeline from another pipeline (Parent).

    Here, whenever any activity in the child pipeline fails the Execute pipeline activity will get failed and it has information about the error message and activity name.

    Follow the below demo for better understanding:

    For sample, I took some set variable activities in child pipeline and intentionally generating error.

    enter image description here

    Now, in the parent pipeline, call this pipeline using Execute pipeline activity and make sure you check on Wait on completion check box.

    On completion of the Execute pipeline, I have added a set variable activity with variable Chat_message. This is the expression to get the chat message.

    @if(equals(activity('Execute Pipeline1').status,'Succeeded'),'The pipeline succeeded',concat('The pipeline failed. The activity name is : ',substring(activity('Execute Pipeline1').error.message, add(indexOf(activity('Execute Pipeline1').error.message,'target'),7), sub(sub(indexOf(activity('Execute Pipeline1').error.message,'failed'),1),add(indexOf(activity('Execute Pipeline1').error.message,'target'),7))),' and the Error message is : ',split(activity('Execute Pipeline1').Error.message,'failed: ')[1]))

    enter image description here

    When any of the child pipeline’s activity gets failed, the total pipeline gets failed and also the Execute pipeline activity in the parent also gets failed.

    Now, on completion of this activity, the above expression, will check whether the pipeline is a failure or success and gives the message as per the status.

    **Message when child pipeline gets failed: **

    enter image description here

    **Message when pipeline succeeded: **

    enter image description here

    As you want to send this message to your team, replace the set variable activity in the parent pipeline with your web activity and in the place of chat message add the above expression. Whether the pipeline is a failure or a success, it uses only single web activity to send the message.

    Login or Signup to reply.
  2. 0

    One way as stated by Rakesh is to leverage Execute pipeline activity but that means managing an additional pipeline.
    You can also manage by adding dependencies in the form of on completion, skip and validate the status of all the activities and get list of all error messages if there are any and send a common email notification.

    enter image description here

    my below blog provides a detailed framework for the same :
    https://datasharkx.wordpress.com/2021/08/19/error-logging-and-the-art-of-avoiding-redundant-activities-in-azure-data-factory/

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search