I'm working on a data flow to ingest XML files and trying to pull a deeply nested node's value up into another part of the flow. E.g. <payload> <header> <customerID>1234</customerID> <createdDate>1/2/2023T15:33:22</createdDate> .... </header> <body> ... <compartments> <compartment> <compartmentNo>1</compartmentNo> <product>2</product> <quantity>5000</quantity>…
I know there are few ways to copy the containers from one subscription storage account to another storage account. using azcopy tool but I do not have access to SAS to generate and use for above activity Through Storage explorer,…
I have a Data Factory (live mode) with a KeyVault linked service (Test connection: successful): KeyVault linked service When I try to use this linked service in a new linked service to connect to a SQL Server, it fails to…
I need to schedule an Azure data factory pipeline that runs every 10th business day of the month. Saturday & Sunday are not business days. for example, if we exclude Sat & Sun, then 10th Business day for Jun 2023…
I am trying to split my column 'State' by abbreviating only the first 2 letters and also making it uppercase in the final output. As you can see from the example there is Ohio, Wisconsin, and Illinois at the top.…
I have an ADF pipeline with a for each loop on files and performing some activity that should only run for the downtime hours. I am starting it using a trigger and want it to gracefully exit execution after 2…
How to check the status of pipeline A from Pipeline B. I have to pass the status of Pipeline A into If activity in pipeline B I have created a pipeline A which has 3 databricks notebook and on Pipeline…
I'm calling the Activity Runs Rest API from a parent pipeline to capture any errors in a child pipeline. However, no data, of any kind, is being returned in the Value array. Here are my settings. Web activity URL https://management.azure.com/subscriptions/@{pipeline().globalParameters.SubscriptionId}/resourceGroups/@{pipeline().globalParameters.ResourceGrpName}/providers/Microsoft.DataFactory/factories/@{pipeline().DataFactory}/pipelineruns/@{pipeline().RunId}/queryActivityRuns?api-version=2018-06-01…
There is set variable holding the below value and it is of type string. { "variableName": "test", "value": "<?xml version="1.0" encoding="utf-8"?> <fetch version="1.0" output-format="xml-platform" mapping="logical" aggregate="true" distinct="false"> <entity name="account"> <attribute name="ownerid" alias="account" aggregate="count" /> <filter type="and"> <condition attribute="ownerid" operator="not-null" />…
I have a pipeline to delete some folders from ADLS. My folder structure is as below. /raw/MainFolder/SubfolderA/20230430/File.parquet /raw/MainFolder/SubfolderA/20230415/File.parquet /raw/MainFolder/SubfolderA/20230410/File.parquet /raw/MainFolder/SubfolderB/20230430/File.parquet /raw/MainFolder/SubfolderB/20230420/File.parquet /raw/MainFolder/SubfolderB/20230405/File.parquet The pipeline is currently deleting the File.parquet under the date folder. But I need it to delete the…