How to add new secrets (from Azure Key Vault) to the variable group in Azure Devops

If you want to set the pipeline to automatically add the new secrets into the variable group, you need to call the following Azure DevOps REST API in the pipeline job:

1. Call the API "Variablegroups – Get" to get the response body (JSON) of the variable group you want to update. From the response body, you can see it contains the contents like as below.

   {
       "name": "VarGroup",
       "providerData": {
           "serviceEndpointId": "<serviceEndpointId>",
           "vault": "<Azure Key vault name>"
       },
       "type": "AzureKeyVault",
       "variableGroupProjectReferences": [
           {
               "name": "VarGroup",
               "projectReference": {
                   "id": "<project id>",
                   "name": "<project name>"
               }
           }
       ],
       "variables": {
           "<SecVar01>": {
               "enabled": true,
               "isReadOnly": false,
               "isSecret": true
           },
           "<SecVar02>": {
               "enabled": true,
               "isReadOnly": false,
               "isSecret": true
           }
       }
   }
   

   The "<SecVar01>" and "<SecVar02>" are the actual names of the secrets that have been added into the variable group.

2. Update the response body by inserting the new secret object as a member of the "variables" node. For example, add new secret "<SecVar03>".

   {
       "name": "VarGroup",
       "providerData": {
           "serviceEndpointId": "<serviceEndpointId>",
           "vault": "<Azure Key vault name>"
       },
       "type": "AzureKeyVault",
       "variableGroupProjectReferences": [
           {
               "name": "VarGroup",
               "projectReference": {
                   "id": "<project id>",
                   "name": "<project name>"
               }
           }
       ],
       "variables": {
           "<SecVar01>": {
               "enabled": true,
               "isReadOnly": false,
               "isSecret": true
           },
           "<SecVar02>": {
               "enabled": true,
               "isReadOnly": false,
               "isSecret": true
           },
           "<SecVar03>": {
               "enabled": true,
               "isReadOnly": false,
               "isSecret": true
           }
       }
   }
   

3. Then call the API "Variablegroups – Update" and pass the updated JSON body above as the Request Body of this API. Once this API call is succeeded, the new secret "<SecVar03>" is added into the variable group.

EDIT:

Below I will share you with a sample of how to call the related REST API to add secret from Azure Key Vault to the variable group using PowerShell script in Azure Pipelines.

1. Since we run the PowerShell script in pipeline, we can use "System.AccessToken" instead of a user’s PAT as the authorization on the API call. To use this token to update the variable group, you need to ensure that you have assigned the Administrator role to the following two build identities on the Security hub of the variable group. For more details, you can see "Job access tokens".

   • Project Collection Build Service ({Organization Name})
   • {Project Name} Build Service ({Organization Name})

   

2. The PowerShell script looks like below.

   # add-secret-from-Key-vault-to-variable-group.ps1
   
   param (
       [string] $organization,
       [string] $project,
       [string] $groupId,
       [string] $secretName
   )
   
   # Provide the authorization through the HTTP headers.
   $headers = @{Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"}
   
   # Call the REST API "Variablegroups - Get" to get the current definition of the variable group.
   $url_getVarGroup = "https://dev.azure.com/${organization}/${project}/_apis/distributedtask/variablegroups/${groupId}?api-version=7.0"
   $resonse_getVarGroup = Invoke-RestMethod -Uri $url_getVarGroup -Headers $headers
   
   # Set the JSON object of the new secret variable added from Azure Key Vault to the variable group.
   $objNewSecVar = @{
       enabled = $true
       isReadOnly = $false
       isSecret = $true
   }
   
   # Add the JSON object of the new secret variable as a member of the "variables" object in the definition.
   $resonse_getVarGroup.variables | add-member -Name "$secretName" -value $objNewSecVar -MemberType NoteProperty
   
   # Convert the content of the updated definition to JSON type for use in the subsequent API call.
   $body_updateVarGroup = @($resonse_getVarGroup) | ConvertTo-Json -Depth 100
   
   # Call the REST API "Variablegroups - Update" to update the variable group with the new definition.
   $url_updateVarGroup = "https://dev.azure.com/${organization}/_apis/distributedtask/variablegroups/${groupId}?api-version=7.0"
   Invoke-RestMethod -Uri $url_updateVarGroup -Method PUT -Body $body_updateVarGroup -ContentType "application/json" -Headers $headers
   

3. Call the PowerShell script in Pipeline.

     - task: PowerShell@2
       displayName: 'Call PowerShell script'
       env:
         SYSTEM_ACCESSTOKEN: $(System.AccessToken)
       inputs:
         targetType: filePath
         filePath: './PowerShellDemo/add-secret-from-Key-vault-to-variable-group.ps1'
         arguments: '-organization "{organization}" -project "{project}" -groupId {groupId} -secretName "{secretName}"'
         pwsh: true
   

   • Replace {organization} with the actual name of your Azure DevOps organization.
   • Replace {project} with the actual name of the project.
   • Replace {groupId} with the actual id of the variable group.
   • Replace {secretName} with the actual name of the secret needs to add from Azure Key Vault to the variable group.