Question 247132 in Azure Question posted in
The official documentation to get you started can be found here.

Why is running an Azure CLI command in a pipeline super slow?

I am very new to ADO pipelines in Azure. I have a situation where I have the build ID of a pipeline run that created an artifact. I need to download this artifact in a different pipeline. In order to use DownloadPipelineArtifact@2 you have to provide both the pipeline an buildId. the docs

In my case, I only have the buildId and figured I could look up the pipeline via an az command, like so:

- task: AzureCLI@2
    displayName: lookup pipeline ID
    inputs:
      azureSubscription: $(ServicePrincipal)
      scriptType: bash
      scriptLocation: inlineScript
      inlineScript: |
        pipelineId=$(az pipelines runs show --id ${{ parameters.buildId }} --query "definition.id")
        echo build ${{ parameters.buildId }} belongs to pipeline $pipelineId
        echo "##vso[task.setvariable variable=pipelineId]$pipelineId"
    env:
      AZURE_DEVOPS_EXT_PAT: $(System.AccessToken)

This all works as expected, BUT when I run that same az command from my local machine, it returns in < 1 second… When it runs in my pipeline, it takes closer to 1 minute.

enter image description here

what am I doing wrong (if anything) and what could I do to make this command happen faster?

UPDATE 1

Based on jessehouwing’s answer, I updated my step like so (I’m on a Windows-hosted agent, not self-hosted):

parameters:
  buildId: ''

steps:
- bash: |
    pipelineId=$(az pipelines runs show --id $BUILD_ID --query "definition.id")
    echo build $BUILD_ID belongs to pipeline $pipelineId
    echo "##vso[task.setvariable variable=pipelineId]$pipelineId"
  displayName: lookup pipeline ID
  env:
    AZURE_DEVOPS_EXT_PAT: $(System.AccessToken)
    BUILD_ID: ${{ parameters.buildId }}

And it actually ran SLOWER,

enter image description here

enter image description here

So, while using bash still got me the data I needed, it didn’t speed anything up.

UPDATE 2

so, I updated my lookup-pipeline step as you suggested, but it had a typo….and the pipeline ID was never returned:

enter image description here

much to my surprise, the download step did not fail…. it actually downloaded the correct artifact. I tested a redeploy of an older build just to make sure it wasn’t grabbing the "latest" build. Sure enough, from what I can tell….despite the docs saying otherwise, the pipeline doesn’t appear to be required when attempting to download an artifact from a pipeline in the same org. Therefore, I have eliminated the lookup step entirely and the entire pipeline works as expected.

Another tragic case of out of date MS docs

Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    turns out, for my specific case, the docs appear to just be flat out wrong and I don't need to do any of the work I was trying to speed up. that said, big thanks to @jessehouwing in helping me out and opening some issues. I'm sure I'll need to use some AZ commands in a pipeline eventually and anything I can do to speed it up will be nice.


  2. 0

    There is a bug in the current hosted runner image which causes az cli to discover all of the installed modules and extensions on the first call, because this data is lost in the provisioning process. This enumeration takes around 50s.

    I’ve created a pull request to fix this. It may take a while for this to get merged.

    The AzureCLI step adds quite a bit of overhead. It does a number of extra things:

    • check the version of Azure CLI
    • check the version of all installed Azure CLI extensions
    • authenticate against Azure using the service connection

    On top of that az pipelines does a discovery of the Azure DevOps organization you’re connecting to and caches this data (calling OPTIONS $(CollectionUri)/_apis and GET $(CollectionUri)/_apis/ResourceAreas). I wonder whether this call is cached on the server, cause it should rarely ever change.

    Since you’re already passing in the magic AZURE_DEVOPS_EXT_PAT environment variable, you don’t need to authenticate to Azure at all, the pipeline has done that already for you. So you can replace AzureCLI with Bash@2 or script to execute your script straight in plain bash to speed things up.

      - script: |
        pipelineId=$(az pipelines runs show --id $BUILD_ID --query "definition.id")
        echo build $BUILD_ID belongs to pipeline $pipelineId
        echo "##vso[task.setvariable variable=pipelineId]$pipelineId"
    env:
      AZURE_DEVOPS_EXT_PAT: $(System.AccessToken)
      BUILD_ID: ${{ parameters.buildId }}

    ⚠️ I updated the above script to pass the BuildId as an environment variable, it’s much safer to pass values that way than to inline them in the script. Inlining them opens up your script for a Scriptjacking attack. even though parameters. is less prone to these issues, it’s better to just always use the environment to pass things along.

    Disable some time consuming features.

    The Azure ZLI has a few features turned on by default. Some take up a little extra time. These settings won’t win you minutes, but every second counts I’d say.

    # Most of these settings are suggested by running `az init` and picking the automation option.
az config set auto-upgrade.enable=false
az config set core.error_recommendation=off
az config set core.disable_progress_bar=true
az config set core.only_show_errors=true
az config set core.collect_telemetry=false
az config set core.no_color=true
az config set logging.enable_log_file=no
az config set output.show_survey_link=no

    Since az adds quite a big of overhead each call you can save a little extra time by just writing the settings file directly:

        - script: |
        c:
        cd %USERPROFILE%
        md .azure 
        cd .azure
        echo [core] > config
        echo first_run = false >> config
        echo collect_telemetry = false >> config
        echo error_recommendation = off >> config
        echo output = json >> config
        echo only_show_errors = true >> config
        echo no_color = true >> config
        echo disable_progress_bar = true >> config
        echo [auto-upgrade] >> config
        echo enable = False >> config
        echo [logging] >> config
        echo enable_log_file = no >> config
        echo [output] >> config
        echo show_survey_link = no >> config

    In most cases

    The call you’re making might not be needed at all. Azure Pipelines populates a number of pre-defined variables including System.DefinitionId which will contain the value you’re looking for without even making a call back to Azure DevOps.

    And in case you’re using a pipeline trigger you could grab the definition id from the calling pipeline using Build.TriggeredBy.DefinitionId.

    Auto-detect

    Another thing that gets into play is the fact that you’ve only passed in the BuildId, this turns on the auto-detect mode for az devops, you’ll get faster execution by explicitly passing the --org and --project:

    steps:
- pwsh: |
        $pipelineId= & az pipelines runs show --id $env:BUILD_ID --query "definition.id" --organization $env:System_CollectionUri --project $env:System_TeamProject
        write-host "build $env:BUILD_ID belongs to pipeline $pipelineId"
        write-host "##vso[task.setvariable variable=pipelineId]$pipelineId"
  env:
    AZURE_DEVOPS_EXT_PAT: $(System.AccessToken)
    BUILD_ID: $(Build.BuildId)

    Funnily enough ubuntu-latest is still much faster than windows.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search