I'm new to Azure Data Factory and have a requirement that sounds pretty simple but I not able to figure it out. I have a Json file saved in a storage acount container, from which i need to read the…
I have my project file siting in the path /Workspace/folder1/subfolder1/myproject This is how I have structured my project files. Structure of Module folder: enter image description here baseMLPipeline is my entry file where I am importing the file in the…
Using ADF, I've created a linked service to a Blob Storage on Azure. The connection to the linked service works: I added a file to the Blob storage which I can clearly see. The Blob is defined with the Firewalls…
We have synapse pipeline that works fine if manually triggered . But when ran in a schedule has the following error message and fails . I verified the parameter names and the parameter does exist in the definition . Any…
How to get alerts in teams and email when a pipeline fails in ADF in monitor alerts. Is there any way to achieve it?
I am new to ADF, I am trying to connect with SQL Server through SSMS setup from my local system. For this, I have created a keyvault in Azure and stored username and passsword into that. Also I have setup…
We are running Python scripts in Azure Data Factory using Batch Service of Azure Data Factory. We are reading secrets from key vault using Service Principle for which we need to hardcode the Client Id, Client Secret which has the…
I took 2 lookups to read 2 excel files and connected to 2 for each activities and inside i have taken 2 set variable and given @item.id Now i have to eliminate the matching records and only unmatched records has…
I have an ADF pipeline which need to run every 4 hours in Central European Time zone. I figured out how to set the trigger to run the pipeline. But, I don't know to make it run just on Weekdays?…
I'm looking to use Azure Synapse's Copy activity to pull data in a big select that has tables coming from more than a single database. Something like this in traditional SQL using Linked Servers -- Select t1.fielda, t1.fieldb, t2.fieldc, t2.fieldd…