How to put Excel as a sink in Azure Data Factory /Azure mapping dataflow
Its kind of weird that i could not find Excel as a sink. How to write the file to Excel using DataFactory?
Split Json array into table rows using azure data factory
I'm using Azure Data Factory and I have Json files in azure Datalake. Each Json file contain an array of Jsons. I want to copy each Jsons of the array into Azure SQL row. I tried to do it with…
How do I pull the last modified file with data flow in azure data factory?
I have files that are uploaded into an onprem folder daily, from there I have a pipeline pulling it to a blob storage container (input), from there I have another pipeline from blob (input) to blob (output), here is were…
Reading folders with Date format (YYYY-MM) using Azure Data Factory
I have few folders inside the Data lake (Example:Test1 container) that are created every month in this format YYYY-MM (Example:2022-11) and inside this folder I have few set of data files, I want to copy this data files to different…
Azure Data Factory SQL trigger events
I wanted to comment on this post: I want to trigger Azure datafactory pipeline whenever there is a change in Azure SQL database but I don't have enough reputation... The solution that Skin comes up with (SQL DB trigger events)…
Azure Data Factory script indexing for pipeline().parameters that are used multiple times in the query
I am using Azure Data Factory script to create parameterized SQL query. I understand that the Index specifies the position in which the parameter's value should go in the SQL command. However, I don't know how to handle the situation…
Overwrite sql table with new data in Azure Dataflow
Here is my situation. Iam using Alteryx ETL tool where in basically we are appending new records to tableau by using option provided like 'Overwrite the file'. What it does is any data incoming is captured to the target and…
Azure – Insert JSON data into SQL DB using Airflow/python
I extracted data from an API using Airflow. The data is extracted from the API and saved on cloud storage in JSON format. The next step is to insert the data into an SQL DB. I have a few questions:…
Azure – ADF linked service connection string update
I'm using ADF dataflow to process data from CosmosDB. Is there a way to auto update the connection key in linked service when the cosmosDB keys are rotated ?
How to check if files present under a data lake directory are empty using Azure Data Factory?
How to check if files present under a data lake directory are empty using Azure Data Factory? There are multiple files present in the data lake directory and I want to check if these files are empty or not, If…