I have successfully created a CI/CD pipeline for Azure Data Factory with Azure DevOps using yaml templates, see below. This in contrast to creating the same pipelines using Azure DevOps 'Classic Editor' When creating Build/Release pipelines using Classic Editor I…
I am not able to register Self Hosted Integration Runtime on my Machine. This is the error I am getting : The Integration Runtime (Self-hosted) node has encountered an error during registration. Unable to determine the Integration Runtime (Self-hosted) node…
This is the example Request body provided by Microsoft to disable a PowerBi schedule refresh via the Rest API { "value": { "enabled": false } } When I call the API, using a service principal, in ADF, with the same…
Im trying to copy data from SQL Server to CosmosDb. I have created a DataFlow to apply transformations and set the proper json schema for the items in the CosmosDb container. The SQL Server data looks like the following: Id…
I need to generate a unique Guid which contains only numeric and of length 20, for every row in the table. Currently, I am running one lookup activity (name - FetchIDs) to run the Stored procedure. Output of the lookup…
I ma trying to create json file from SQL table using ADF through Stored Procedure Stored Procedue: SELECT ( SELECT * FROM [dbo].[TEST_TABLE] FOR JSON PATH, INCLUDE_NULL_VALUES ) AS json_file; I am then trying copy activity where Source is SQL…
I need to check if particular SQL database table contains variable currentID. Currently, I am running one lookup activity (name - GlobalIDs) to query the SQL table and retrieve the data. Output of the lookup activity GlobalIDs is as shown…
In Azure Data Factory, I have recently moved a CosmosDb Connection which was using a connection string to using Managed Identity with RBAC (Cosmos DB Built-in Data Reader role). However, I noticed early on in the pipeline, when there are…
I am trying to split the item name if the item name contains '_' and if those item name starts with a specific schema names item = ['test_tabl','Schema1__mytable1', 'Schema2__mytable2'] @concat('SELECT *FROM ', if(contains(item(), '_'), and(startswith(item(), 'Schema1_'),item()), or(startswith(item(), 'Schema2_'),item()), concat(split(item(), '_')[0],…
I've been struggling to find a straightforward solution for connecting Azure data factory or Fabric to an on-premises SSAS cube. I've searched extensively, but all I come across are guides on connecting to SQL Server or discussions stating that connecting…