I developed a CI/CD to deploy the content (pipelines, Global parameters, LinkedServices…) of a DEV Data Factory in a PROD Data Factory.
I use 2 shirs for the DEV and PROD environments where each environment uses its own shir.
When I publish content from the DEV Data Factory which is connected to an Azure DevOps repository, I generate the ARMTemplateForFactory.json and ARMTemplateParametersForFactory.json files
The problem is that the name of the shir, by default is not a parameter of the ARMTemplateParametersForFactory.json file
I would like to include the name of the integration runtime (shir) used in each environment as a parameter of the generated ARM Template when publishing.
After some research I found that you could modify the structure of the arm-template-parameters-definition.json file to customize the settings of the ARM Template. However, I don’t know what Syntax used to do so: /
So my question is: how to include the name of the integration runtime as a parameter of the ARM Template.
Thanks a lot !
This is the json description of my integration runtime :
{
"name": "TestRuntimeSam",
"properties": {
"type": "Managed",
"typeProperties": {
"computeProperties": {
"location": "AutoResolve",
"dataFlowProperties": {
"computeType": "General",
"coreCount": 8,
"timeToLive": 10,
"cleanup": false
}
}
}
}
}
and this is the basic content of the arm-template-parameters-definition.json file :
{
"Microsoft.DataFactory/factories": {
"properties": {
"globalParameters": {
"*": {
"value": "="
}
},
"globalConfigurations": {
"*": "="
},
"encryption": {
"*": "=",
"identity": {
"*": "="
}
}
},
"location": "=",
"identity": {
"type": "=",
"userAssignedIdentities": "="
}
},
"Microsoft.DataFactory/factories/pipelines": {},
"Microsoft.DataFactory/factories/integrationRuntimes": {
"properties": {
"typeProperties": {
"ssisProperties": {
"catalogInfo": {
"catalogServerEndpoint": "=",
"catalogAdminUserName": "=",
"catalogAdminPassword": {
"value": "-::secureString"
}
},
"customSetupScriptProperties": {
"sasToken": {
"value": "-::secureString"
}
}
},
"linkedInfo": {
"key": {
"value": "-::secureString"
},
"resourceId": "="
},
"computeProperties": {
"dataFlowProperties": {
"externalComputeInfo": [
{
"accessToken": "-::secureString"
}
]
}
}
}
}
},
"Microsoft.DataFactory/factories/triggers": {
"properties": {
"pipelines": [
{
"parameters": {
"*": "="
}
},
"pipelineReference.referenceName"
],
"pipeline": {
"parameters": {
"*": "="
}
},
"typeProperties": {
"scope": "="
}
}
},
"Microsoft.DataFactory/factories/linkedServices": {
"*": {
"properties": {
"typeProperties": {
"accountName": "=",
"accountEndpoint": "=",
"username": "=",
"userName": "=",
"accessKeyId": "=",
"endpoint": "=",
"servicePrincipalId": "=",
"userId": "=",
"host": "=",
"clientId": "=",
"existingClusterId": "=",
"clusterUserName": "=",
"clusterSshUserName": "=",
"hostSubscriptionId": "=",
"clusterResourceGroup": "=",
"subscriptionId": "=",
"resourceGroupName": "=",
"tenant": "=",
"dataLakeStoreUri": "=",
"baseUrl": "=",
"database": "=",
"serviceEndpoint": "=",
"batchUri": "=",
"poolName": "=",
"databaseName": "=",
"systemNumber": "=",
"server": "=",
"url": "=",
"functionAppUrl": "=",
"environmentUrl": "=",
"aadResourceId": "=",
"sasUri": "|:-sasUri:secureString",
"sasToken": "|",
"connectionString": "|:-connectionString:secureString",
"hostKeyFingerprint": "=",
"mlWorkspaceName": "="
}
}
},
"Odbc": {
"properties": {
"typeProperties": {
"userName": "=",
"connectionString": {
"secretName": "="
}
}
}
}
},
"Microsoft.DataFactory/factories/datasets": {
"*": {
"properties": {
"typeProperties": {
"folderPath": "=",
"fileName": "="
}
}
}
},
"Microsoft.DataFactory/factories/credentials": {
"*": {
"properties": {
"typeProperties": {
"token": "="
}
}
}
},
"Microsoft.DataFactory/factories/managedVirtualNetworks/managedPrivateEndpoints": {
"properties": {
"privateLinkResourceId": "=",
"groupId": "=",
"fqdns": "="
}
},
"Microsoft.DataFactory/factories/globalparameters": {
"properties": {
"*": {
"value": "="
}
}
}
}
What shall I add ?
2
Answers
Please open the {} symbol in from of the IR name and look through the properties
and then add them in the template under a section as shown in the screenshot below:
Please use this link : https://patrick-picard.medium.com/azure-data-factory-modifying-arm-template-parameters-53b11f38bced
I appreciate this isn’t precisely the answer you were looking for (you wanted to know what to put in the parameters file). I couldn’t find that answer when I looked, I don’t think it’s possible, unless something has changed recently. What I did find, is that you can add a Powershell Script, at the start of your Deployment pipeline, that does a find and replace in your Template File, and replaces the name. It doesn’t seem ideal, but it works. See the answer given here:
https://learn.microsoft.com/en-us/answers/questions/776353/how-can-we-parametrize-two-different-integration-r