I am working on ADF data flow and I am trying to implement a logic for two step sorting in my source data. The requirement is like, I have 1 primary key and 2 date columns : example- id, date1…
I have a copy activity in Data Factory which copies data from 1 SQL server to an other one. Tables are identical. Most of the data is copied correctly but starting this week some columns are 0 instead of the…
I am passing some override parameters to my Data Factory ARM template deployment step in an Azure DevOps yaml file and one global parameter in particular is specified as an 'int' type within the generated ARM template. When I attempt…
I copy data from postgre db to azure sql with copy activity and auto-create table feature enabled. there is a column with type timestamptz in Source db. looks like 2023-07-03T07:00:47.707369+00:00 2023-07-03T07:00:52.375166+00:00 when I auto-create table in Azure SQL it always…
I am an experienced Azure user when it comes to data infrastructure and pipelines. However, I am new to processing XML data and my particular case seems somewhat complex. After exhausting my resources I am turning to the community for…
I'm working on a data flow to ingest XML files and trying to pull a deeply nested node's value up into another part of the flow. E.g. <payload> <header> <customerID>1234</customerID> <createdDate>1/2/2023T15:33:22</createdDate> .... </header> <body> ... <compartments> <compartment> <compartmentNo>1</compartmentNo> <product>2</product> <quantity>5000</quantity>…
I know there are few ways to copy the containers from one subscription storage account to another storage account. using azcopy tool but I do not have access to SAS to generate and use for above activity Through Storage explorer,…
I have a Data Factory (live mode) with a KeyVault linked service (Test connection: successful): KeyVault linked service When I try to use this linked service in a new linked service to connect to a SQL Server, it fails to…
I need to schedule an Azure data factory pipeline that runs every 10th business day of the month. Saturday & Sunday are not business days. for example, if we exclude Sat & Sun, then 10th Business day for Jun 2023…
I am trying to split my column 'State' by abbreviating only the first 2 letters and also making it uppercase in the final output. As you can see from the example there is Ohio, Wisconsin, and Illinois at the top.…