How to replace Source with other Source in Azure Data Flow
As you can see Number 1 Source has to be Replaced with Number 2 Union Source .There is no option to connect the same line of transformations to new source .How can i achieve this
Azure – How to do two step sorting based on condition in dataflow ADF
I am working on ADF data flow and I am trying to implement a logic for two step sorting in my source data. The requirement is like, I have 1 primary key and 2 date columns : example- id, date1…
Azure – Copy activitity Data Factory give 0 on some columns
I have a copy activity in Data Factory which copies data from 1 SQL server to an other one. Tables are identical. Most of the data is copied correctly but starting this week some columns are 0 instead of the…
Ubuntu – Deploying ARM Template with override parameters in Azure DevOps is converting int values as string type
I am passing some override parameters to my Data Factory ARM template deployment step in an Azure DevOps yaml file and one global parameter in particular is specified as an 'int' type within the generated ARM template. When I attempt…
Postgresql – Azure Data Factory set DateTimeOffset data type for auto-created table in Copy activity
I copy data from postgre db to azure sql with copy activity and auto-create table feature enabled. there is a column with type timestamptz in Source db. looks like 2023-07-03T07:00:47.707369+00:00 2023-07-03T07:00:52.375166+00:00 when I auto-create table in Azure SQL it always…
XML Data to Azure SQL Server via Azure Data Factory V2
I am an experienced Azure user when it comes to data infrastructure and pipelines. However, I am new to processing XML data and my particular case seems somewhat complex. After exhausting my resources I am turning to the community for…
Azure Data Flows – Accessing deeply nested values in an XML file in
I'm working on a data flow to ingest XML files and trying to pull a deeply nested node's value up into another part of the flow. E.g. <payload> <header> <customerID>1234</customerID> <createdDate>1/2/2023T15:33:22</createdDate> .... </header> <body> ... <compartments> <compartment> <compartmentNo>1</compartmentNo> <product>2</product> <quantity>5000</quantity>…
Azure – Unable to copy the multiple containers from one subscription storage account to another subscription another storage account
I know there are few ways to copy the containers from one subscription storage account to another storage account. using azcopy tool but I do not have access to SAS to generate and use for above activity Through Storage explorer,…
Azure Data Factory linked service not working using KeyVault linked service – Secret "NotFound"
I have a Data Factory (live mode) with a KeyVault linked service (Test connection: successful): KeyVault linked service When I try to use this linked service in a new linked service to connect to a SQL Server, it fails to…
How to schedule an azure datafactory pipeline to run on every 'n'th business day of a month
I need to schedule an Azure data factory pipeline that runs every 10th business day of the month. Saturday & Sunday are not business days. for example, if we exclude Sat & Sun, then 10th Business day for Jun 2023…