Backslash from the json is getting removed after the file is processed by copy activity I have json input file which contains the below object INPUT { "Name": "xyz", "DOB":"14/09/1995" } After i read the files and save using the…
I would like to copy some files from one storage account to another. Also, I would like to set the access tier of the target files to cold. I tried to do this with the Azure Data Factory as described…
I have a copy activity to copy a csv file from one container in the data lake to another container in the data lake. Source and sink datasets are binary format type with identical settings. I can't see rowsCopied in…
In Azure on Subscription level I have **Azure Data Factory Contributor **Role and on the Resource Group I'm the owner. With this permission set I still can't create an Azure Data Factory in the Resource Group. What role/permission do I…
I have an array output (len = 503 ) from the lookup value activity which is basically fetching strings from a CSV file. These strings are parameters for my dataflow activity within the for each. Current adf pipeline: I now…
There is a variable in ADF that holds the below value. @json(concat('{','"S.No":',variables('counter'),',"Input Table":"Input TblName","Output Table":"[Tbl1, tbl2, Tbl3]", "Status":"Successful"}')) I pass this variable to a logic app using Web activity and use 'Create HTML Table' to convert this json to a…
Is there a easy way to access azure blob metadata key values through data factory or any other easier solution when loading the data in to azure dataware house on fabric. I tried using getmetadata component and looks like that…
I am running an incremental process for multiple tables where I am using old watermark, new watermark, copy activity, and a stored procedure to update the watermark value. One of my table succeeds when I run the pipeline, for eg…
We have a source table with 1M records. Source, Target DB is Azure DB for SQL Server. We do not want all 1 M rows in target database. Target table works as a cache table for us. Every fortnight, we…
In DataFactory, I'm creating a ForEach activity and in Items I'm using this expression: @createArray(json('{ "daily": { "raw": [ { "entity_name": "ent1", "fetchXML_file": "raw/ent1.xml", "destination_path_base": "/master/raw/ent1" }, { "entity_name": "ent2", "fetchXML_file": "raw/ent2.xml", "destination_path_base": "/master/raw/ent2" }, { "entity_name": "ent3", "fetchXML_file": "raw/ent3.xml",…