I ma trying to create json file from SQL table using ADF through Stored Procedure Stored Procedue: SELECT ( SELECT * FROM [dbo].[TEST_TABLE] FOR JSON PATH, INCLUDE_NULL_VALUES ) AS json_file; I am then trying copy activity where Source is SQL…
I need to check if particular SQL database table contains variable currentID. Currently, I am running one lookup activity (name - GlobalIDs) to query the SQL table and retrieve the data. Output of the lookup activity GlobalIDs is as shown…
In Azure Data Factory, I have recently moved a CosmosDb Connection which was using a connection string to using Managed Identity with RBAC (Cosmos DB Built-in Data Reader role). However, I noticed early on in the pipeline, when there are…
I am trying to split the item name if the item name contains '_' and if those item name starts with a specific schema names item = ['test_tabl','Schema1__mytable1', 'Schema2__mytable2'] @concat('SELECT *FROM ', if(contains(item(), '_'), and(startswith(item(), 'Schema1_'),item()), or(startswith(item(), 'Schema2_'),item()), concat(split(item(), '_')[0],…
I've been struggling to find a straightforward solution for connecting Azure data factory or Fabric to an on-premises SSAS cube. I've searched extensively, but all I come across are guides on connecting to SQL Server or discussions stating that connecting…
To start, it seems like pagination works differently when using a Copy data command vs. using a Data Flow to do the same thing, but adding a flatten transformation inbetween the ingest and sink. When using the Copy data command,…
I have a file which is present as a unstructured txt in ADLS. I want to read the entire file content as text and pass it to a Stored Procedure in Azure data factory. Can this be done through Azure…
I'm trying to copy data from Salesforce to Snowflake using ADF avoiding use SAS Token, because of that I going to use integration storage on Snowflake. I'm trying to follow this example: https://medium.com/snowflake/seamless-migration-to-snowflake-using-adf-script-activity-schema-detection-25475ea86a09. I already create the integration storage, format…
This is my input file - It is just a sample It will have more than 20k rows {"A":12345,"B":"2023-12-31 00:00:00","C":5510,"D":20231231,"E":20240103,"F":20231231,"G":1577158,"H":63217,"I":null,"J":"Company123","K":"SupplierXYZ","L":"987","M":"Vendor ABC","N":"Invoice789","O":"2023-12-31T00:00:00","P":"2024-01-03T00:00:00","Q":"2023033344","R":"7559","S":"Dept123","T":"ModelXYZ","U":7179.0,"V":3,"W":"3769686.0","X":"TypeJ","Y":7179.0,"Z":null,"AA":"NOK","AB":"202312.0","AC":"EXTERNAL","AD":null,"AE":"Counterparty999","AF":"2023-12-31T00:00:00","AG":null,"AH":null,"AI":"66c6c4cc31107992a4079f3653a8a86930a2675d8663c5ab2ec4df23b0f6f3b0","AJ":"2024-02-13T12:58:02.597","AK":"2024-01-03T08:31:16.425292"} {"A":54321,"B":"2024-01-15 08:00:00","C":7890,"D":20240115,"E":20240120,"F":20240115,"G":2468135,"H":951753,"I":null,"J":"Company456","K":"Supplier789","L":"123456789","M":"Vendor XYZ","N":"Invoice123","O":"2024-01-15T08:00:00","P":"2024-01-20T08:00:00","Q":"2024011515","R":"1234","S":"Dept456","T":"ModelABC","U":9876.0,"V":6,"W":"9876543.0","X":"TypeK","Y":9876.0,"Z":null,"AA":"USD","AB":"202401.0","AC":"INTERNAL","AD":null,"AE":"Counterparty888","AF":"2024-01-15T08:00:00","AG":null,"AH":null,"AI":"a1b2c3d4e5f6g7h8i9","AJ":"2024-02-15T08:00:00.123","AK":"2024-01-20T08:00:00.987654"} I want the output as { "dataRepositoryId": "fixedvalue", "importName": "uuid, "data": [ {"A":12345,"B":"2023-12-31 00:00:00","C":5510,"D":20231231,"E":20240103,"F":20231231,"G":1577158,"H":63217,"I":null,"J":"Company123","K":"SupplierXYZ","L":"987","M":"Vendor ABC","N":"Invoice789","O":"2023-12-31T00:00:00","P":"2024-01-03T00:00:00","Q":"2023033344","R":"7559","S":"Dept123","T":"ModelXYZ","U":7179.0,"V":3,"W":"3769686.0","X":"TypeJ","Y":7179.0,"Z":null,"AA":"NOK","AB":"202312.0","AC":"EXTERNAL","AD":null,"AE":"Counterparty999","AF":"2023-12-31T00:00:00","AG":null,"AH":null,"AI":"66c6c4cc31107992a4079f3653a8a86930a2675d8663c5ab2ec4df23b0f6f3b0","AJ":"2024-02-13T12:58:02.597","AK":"2024-01-03T08:31:16.425292"}, {"A":54321,"B":"2024-01-15…
I have a dataset setup in my ADF solution which points to a blob container. There are 25 CSV files sitting in this container. I have a dataflow setup in ADF which currently points to each .CSV file respectfully. For…