In our ADF application pipeline A is invoking pipeline B. I want to capture the run ID so I used the @pipeline().RunId to capture it. But while executing the two run ID Which is as per design. But I want…
Its kind of weird that i could not find Excel as a sink. How to write the file to Excel using DataFactory?
I'm using Azure Data Factory and I have Json files in azure Datalake. Each Json file contain an array of Jsons. I want to copy each Jsons of the array into Azure SQL row. I tried to do it with…
I have files that are uploaded into an onprem folder daily, from there I have a pipeline pulling it to a blob storage container (input), from there I have another pipeline from blob (input) to blob (output), here is were…
I have few folders inside the Data lake (Example:Test1 container) that are created every month in this format YYYY-MM (Example:2022-11) and inside this folder I have few set of data files, I want to copy this data files to different…
I wanted to comment on this post: I want to trigger Azure datafactory pipeline whenever there is a change in Azure SQL database but I don't have enough reputation... The solution that Skin comes up with (SQL DB trigger events)…
I am using Azure Data Factory script to create parameterized SQL query. I understand that the Index specifies the position in which the parameter's value should go in the SQL command. However, I don't know how to handle the situation…
Here is my situation. Iam using Alteryx ETL tool where in basically we are appending new records to tableau by using option provided like 'Overwrite the file'. What it does is any data incoming is captured to the target and…
I extracted data from an API using Airflow. The data is extracted from the API and saved on cloud storage in JSON format. The next step is to insert the data into an SQL DB. I have a few questions:…
I'm using ADF dataflow to process data from CosmosDB. Is there a way to auto update the connection key in linked service when the cosmosDB keys are rotated ?