Passing Variable Between Stages in Azure DevOps Pipeline

Killnine
November 12, 2024
111 views
0 votes
2 Answers

I have a build pipeline that needs to create a URL with a Shared Access Signature (see: SAS or authentication key) for access to a ZIP file downstream in the pipeline. I am able to create the URL and SAS key just fine but getting it to another stage isn’t working.

The process is documented fairly well here but I must be missing something.

I’ve included the full pipeline here for context, but the main logic is in the Generate SAS URL for the artifact task and Deployment to prod job:

trigger:
- main

variables:
  - group: xyz

pool:
  name: Default

stages:
- stage: build
  displayName: Build the application

  jobs:
    - job: build
      displayName: Build
      steps:
        - task: UseDotNet@2
          inputs:
            version: 8.x
        - task: DotNetCoreCLI@2
          displayName: Build Function App
          inputs:
            command: 'build'
            projects: |
              $(System.DefaultWorkingDirectory)/src/AzureFunctions/AzureFunctions.csproj
            arguments: --output $(System.DefaultWorkingDirectory)/function-package --configuration Release

        - task: ArchiveFiles@2
          displayName: 'Archive files'
          inputs:
            rootFolderOrFile: '$(System.DefaultWorkingDirectory)/function-package'
            includeRootFolder: false
            archiveType: zip
            archiveFile: $(Build.ArtifactStagingDirectory)/$(Build.BuildNumber).zip
            replaceExistingArchive: true

        - publish: $(Build.ArtifactStagingDirectory)/$(Build.BuildNumber).zip
          artifact: function-package

        - task: AzureCLI@2
          displayName: 'Upload artifact to Azure Storage'
          inputs:
            azureSubscription: '<secret>'
            scriptType: 'bash'
            scriptLocation: 'inlineScript'
            inlineScript: |
              storage_key=$(az storage account keys list --resource-group <secret> --account-name <secret> --query '[0].value' --output tsv)
              az storage blob upload 
                --account-name <secret> 
                --container-name function-artifacts/<secret> 
                --name $(Build.BuildNumber).zip 
                --file $(Build.ArtifactStagingDirectory)/$(Build.BuildNumber).zip 
                --account-key $storage_key

        - task: AzureCLI@2
          displayName: 'Generate SAS URL for the artifact'
          inputs:
            azureSubscription: '<secret>'
            scriptType: 'bash'
            scriptLocation: 'inlineScript'
            inlineScript: |
              storage_key=$(az storage account keys list --resource-group <secret> --account-name <secret> --query '[0].value' --output tsv)
              sas_url=$(az storage blob generate-sas 
                --account-name <secret> 
                --container-name function-artifacts/<secret> 
                --name $(Build.BuildNumber).zip 
                --permissions r 
                --expiry $(date -u -d "1 year" '+%Y-%m-%dT%H:%MZ') 
                --account-key $storage_key 
                --output tsv)
              echo "##vso[task.setvariable variable=ArtifactSasUrl;isOutput=true]https://<secret>.blob.core.windows.net/function-artifacts/<secret>/$(Build.BuildNumber).zip?$sas_url"

        - publish: $(System.DefaultWorkingDirectory)/build/bicep
          artifact: bicep-package

- stage: deployment_prod
  dependsOn: build
  displayName: Deployment to prod
  variables:
    ArtifactSasUrl: $[dependencies.build.outputs['build.ArtifactSasUrl']]
  jobs:
    - template: deployment.yaml
      parameters:
        Environment: 'prod'
        StorageAccount: $(StorageAccount)
        LoftwareDb: $(<secretDb>)
        AppServicePlanResourceId: $(AppServicePlanResourceId)
        ResourceGroupName: '<secret>'
        ServiceConnectionName: '<secret>'
        SubscriptionId: '<secret>'
        ArtifactUrl: '$(ArtifactSasUrl)'

I have a step in a downstream process that prints out the SAS URL and it’s empty (no error, just empty).

Answers

To access output variable you need to nake step first

        - task: AzureCLI@2
          name: generateSasUrl
          displayName: 'Generate SAS URL for the artifact'
          inputs:
            azureSubscription: '<secret>'
            scriptType: 'bash'
            scriptLocation: 'inlineScript'
            inlineScript: |
              storage_key=$(az storage account keys list --resource-group <secret> --account-name <secret> --query '[0].value' --output tsv)
              sas_url=$(az storage blob generate-sas 
                --account-name <secret> 
                --container-name function-artifacts/<secret> 
                --name $(Build.BuildNumber).zip 
                --permissions r 
                --expiry $(date -u -d "1 year" '+%Y-%m-%dT%H:%MZ') 
                --account-key $storage_key 
                --output tsv)
              echo "##vso[task.setvariable variable=ArtifactSasUrl;isOutput=true]https://<secret>.blob.core.windows.net/function-artifacts/<secret>/$(Build.BuildNumber).zip?$sas_url"

and then:

    ArtifactSasUrl: $[dependencies.build.outputs['build.generateSasUrl.ArtifactSasUrl']]

Here is the JSON map for dependencies

"dependencies": {
  "<STAGE_NAME>" : {
    "result": "Succeeded|SucceededWithIssues|Skipped|Failed|Canceled",
    "outputs": {
        "jobName.stepName.variableName": "value"
    }
  },
  "...": {
    // another stage
  }
}

As suggested by @KrzysztofMadej, you should add the name: generateSasUrl property for your AzureCLI@2 task to Generate SAS URL for the artifact to reference the step that outputs the variable ArtifactSasUrl in the dependent stage.

Based on the tests from my side, we need to single quote '$sas_url' in order pass a valid SAS token rather than a broken one.

In the dependent stage of deployment_prod, we should use stageDependencies syntax to reference the variable $(ArtifactSasUrl), like the sample in this document to Set an output variable for use in future stages. Also we can use the expression $[convertToJson(stageDependencies)] to expand stageDependencies object context for troubleshooting.

With those being said, here is my working YAML pipeline for your reference.

stages:
- stage: build
  displayName: Build the application
  jobs:
    - job: build
      displayName: Build
      steps:
        ...

        - task: ArchiveFiles@2
          displayName: 'Archive files'
          inputs:
            rootFolderOrFile: '$(System.DefaultWorkingDirectory)'
            includeRootFolder: false
            archiveType: zip
            archiveFile: $(Build.ArtifactStagingDirectory)/$(Build.BuildNumber).zip
            replaceExistingArchive: true
        - publish: $(Build.ArtifactStagingDirectory)/$(Build.BuildNumber).zip
          artifact: function-package
        - task: AzureCLI@2
          displayName: 'Upload artifact to Azure Storage'
          inputs:
            azureSubscription: '$(ARMSvcCnn)'
            scriptType: 'bash'
            scriptLocation: 'inlineScript'
            inlineScript: |
              storage_key=$(az storage account keys list --resource-group $(RG) --account-name $(StorageAccount) --query '[0].value' --output tsv)
              az storage blob upload 
                --account-name $(StorageAccount) 
                --container-name function-artifacts/test 
                --name $(Build.BuildNumber).zip 
                --file $(Build.ArtifactStagingDirectory)/$(Build.BuildNumber).zip 
                --account-key $storage_key

        - task: AzureCLI@2
          name: generateSasUrl
          displayName: 'Generate SAS URL for the artifact'
          inputs:
            azureSubscription: '$(ARMSvcCnn)'
            scriptType: 'bash'
            scriptLocation: 'inlineScript'
            inlineScript: |
              storage_key=$(az storage account keys list --resource-group $(RG) --account-name $(StorageAccount) --query '[0].value' --output tsv)
              sas_url=$(az storage blob generate-sas 
                --account-name $(StorageAccount) 
                --container-name function-artifacts/test 
                --name $(Build.BuildNumber).zip 
                --permissions r 
                --expiry $(date -u -d "1 year" '+%Y-%m-%dT%H:%MZ') 
                --account-key $storage_key 
                --output tsv)
              echo "Checking sas_url - $sas_url"
              echo "##vso[task.setvariable variable=SasUrl;isOutput=true]'$sas_url'"
              echo "##vso[task.setvariable variable=ArtifactSasUrl;isOutput=true]https://$(StorageAccount).blob.core.windows.net/function-artifacts/test/$(Build.BuildNumber).zip?'$sas_url'"
        - script: |
            echo $(generateSasUrl.ArtifactSasUrl)
            echo $(generateSasUrl.SasUrl)

- stage: deployment_prod
  dependsOn: build
  displayName: Deployment to prod
  variables:
    ArtifactSasUrl: $[stageDependencies.build.build.outputs['generateSasUrl.ArtifactSasUrl']]
    stageDeps: $[convertToJson(stageDependencies)]
  jobs:
  - job: downstreamJob
    steps:
    - script: |
        echo "$(ArtifactSasUrl)"
        echo "$(stageDeps)"

Please signup or login to give your own answer.

Click here to cancel reply.