Azure Question posted in
The official documentation to get you started can be found here.

Azure – CI/CD Pipeline Skips Deployments in Multi-Environment Setup

I am building a CI/CD pipeline with the following steps:

  • Check for any changes.
  • If changes are found, deploy.

This setup is intended for multiple environments. However, I am encountering an issue when there are more than one environment to deploy:

The deployment for the first environment from the template works as expected. However, the second environment, despite detecting changes, does not initiate the deployment and is marked as "skipped". First env always work like a charm:
enter image description here

Here’s a simplified version of my pipeline configuration:

trigger:
  batch: true
  branches:
    include:
      - main

pr:
  branches:
    include:
      - main

stages:
- template: templates/deploy.yml
  parameters:
    environment: Dev

- template: templates/deploy.yml
  parameters:
    environment: Uat

and deploy.yml template:

stages:
- stage: Terraform_Plan_${{ parameters.environment }}
  displayName: Plan
  condition: always()
  jobs:
  - job: Terraform_Plan_${{ parameters.environment }}
    displayName: Plan Terraform
    pool: 'selfhosted'
    steps:

    - powershell: |
          # set it to true or false
          Write-Host "##vso[task.setvariable variable=anyTfChanges;isOutput=true]true"

      displayName: Detect any Terraform changes
      name: anyTfChanges


- stage: Any_Tf_Changes_${{ parameters.environment }}
  displayName: Terraform Changes
  dependsOn: Terraform_Plan_${{ parameters.environment }}
  variables:
     anyTfChanges: $[ stageDependencies.Terraform_Plan_${{ parameters.environment }}.Terraform_Plan_${{ parameters.environment }}.outputs['anyTfChanges.anyTfChanges'] ]
  condition: and(eq('true', 'true'), eq(dependencies.Terraform_Plan_${{ parameters.environment }}.outputs['Terraform_Plan_${{ parameters.environment }}.anyTfChanges.anyTfChanges'], 'true'))
 
  jobs:
    - job: Terraform_Changes_${{ parameters.environment }}
      displayName: Detect Terraform Changes
      steps:
      - checkout: none
      - powershell: |
          Write-Host "hello world"
        displayName: Terraform changes detected

- stage: Terraform_Apply_${{ parameters.environment }}
  displayName: Apply ${{ parameters.environment }}
  dependsOn: Any_Tf_Changes_${{ parameters.environment }}
  jobs:
  - deployment: Apply
    environment: ${{ parameters.environmentDisplayName }}
    displayName: Apply Terraform
    pool: 'selfhosted'
    strategy:
      runOnce:
        deploy:
          steps:
          - checkout: self
          - script: |
              Write-Host "hello world"

Could I be overlooking something in my configuration that causes the pipeline to behave this way for subsequent environments? Any insights or suggestions would be appreciated!

EDIT1

It only works for first stage:
enter image description here

And when next one has changes. For third one (prod) it does not work when middle stage doesnt have changes

enter image description here

Also when i have two stages, and no changes in first one i can not deploy to second stage because apply stage is skipped

EDIT2

When i commented depends on: in apply stage, it still does not work:
enter image description here

Tags:

2

Answers


  1. 0

    From Define conditions:

    By default, a stage runs if it doesn’t depend on any other stage, or
    if all of the stages that it depends on have completed and
    succeeded    . You can customize this behavior by forcing a stage to run
    even if a previous stage fails or by specifying a custom condition.

    The Apply Prd stage has no conditions set and some of the previous stages were skipped, so that’s why the stage was skipped:

    Pipeline stages

    Fixing the pipeline

    The below example pipeline was used to test the dependencies between stages.

    Set applyChanges value to true or false to simulate changes in each environment:

    azure-pipeline.yml

    trigger: none

pool:
  vmImage: 'ubuntu-latest'

stages:
  - template: templates/deploy.yml
    parameters:
      environment: 'Dev'
      applyChanges: 'true'
  
  - template: templates/deploy.yml
    parameters:
      environment: Uat
      applyChanges: 'false'
  
  - template: templates/deploy.yml
    parameters:
      environment: Prod
      applyChanges: 'true'

    templates/deploy.yml

    parameters:
- name: environment
  displayName: Environment
  type: string

- name: applyChanges
  displayName: Apply Changes
  type: string

- name: environmentDisplayName
  displayName: Environment Display Name
  type: string
  default: my-environment # change it to an existing environment

stages:
- stage: plan_stage_${{ parameters.environment }}
  displayName: Plan ${{ parameters.environment }}
  condition: always()
  jobs:
  - job: plan_job # no need to have distinct job names for jobs running in different stages
    displayName: Plan Terraform ${{ parameters.environment }}
    steps:
    - checkout: none
    - powershell: |
          # set it to true or false
          Write-Host "anyTfChanges=${{ parameters.applyChanges }}"
          Write-Host "##vso[task.setvariable variable=anyTfChanges;isOutput=true]${{ parameters.applyChanges }}"
      displayName: Detect any Terraform changes
      name: detectChanges


- stage: detect_changes_stage_${{ parameters.environment }}
  displayName: Terraform Changes ${{ parameters.environment }}
  dependsOn: plan_stage_${{ parameters.environment }}
  # condition syntax: dependencies.STAGE_NAME.outputs['JOB_NAME.TASK_NAME.VARIABLE_NAME']
  condition: eq(dependencies.plan_stage_${{ parameters.environment }}.outputs['plan_job.detectChanges.anyTfChanges'], 'true')
  # OR
  # previous stage succeeded and there are changes (anyTfChanges=true)
  # condition: and(succeeded('plan_stage_${{ parameters.environment }}'), eq(dependencies.plan_stage_${{ parameters.environment }}.outputs['plan_job.detectChanges.anyTfChanges'], 'true'))
  variables:
    # variable syntax: stageDependencies.STAGE_NAME.JOB_NAME.outputs['TASK_NAME.VARIABLE_NAME']
    anyTfChanges: $[ stageDependencies.plan_stage_${{ parameters.environment }}.plan_job.outputs['detectChanges.anyTfChanges'] ]
  jobs:
    - job: detect_changes_job # no need to have distinct job names for jobs running in different stages
      displayName: Detect Changes ${{ parameters.environment }}
      steps:
      - checkout: none
      - powershell: |
          Write-Host "anyTfChanges: $(anyTfChanges)"
        displayName: Terraform changes detected

- stage: apply_stage_${{ parameters.environment }}
  displayName: Apply ${{ parameters.environment }}
  dependsOn: detect_changes_stage_${{ parameters.environment }}
  # condition syntax: dependencies.STAGE_NAME.outputs['JOB_NAME.TASK_NAME.VARIABLE_NAME']
  condition: eq(dependencies.plan_stage_${{ parameters.environment }}.outputs['plan_job.detectChanges.anyTfChanges'], 'true')
  # OR
  # previous stage succeeded and there are changes (anyTfChanges=true)
  # condition: and(succeeded('detect_changes_stage_${{ parameters.environment }}'), eq(dependencies.plan_stage_${{ parameters.environment }}.outputs['plan_job.detectChanges.anyTfChanges'], 'true'))
  variables:
    # variable syntax: stageDependencies.STAGE_NAME.JOB_NAME.outputs['TASK_NAME.VARIABLE_NAME']
    anyTfChanges: $[ stageDependencies.plan_stage_${{ parameters.environment }}.plan_job.outputs['detectChanges.anyTfChanges'] ]
  jobs:
  - deployment: Apply
    environment: ${{ parameters.environmentDisplayName }}
    displayName: Apply Changes ${{ parameters.environment }}
    strategy:
      runOnce:
        deploy:
          steps:
          - checkout: none
          - powershell: |
              Write-Host "anyTfChanges: $(anyTfChanges)"
            displayName: Apply changes

    Please note that I’ve changed the names of the stages, jobs and tasks to make it easier to understand.

    Running the pipeline:

    Pipeline run

    Login or Signup to reply.
  2. 0

    Based on the documents Conditions – Azure Pipelines | Microsoft Learn and Define variables – Azure Pipelines | Microsoft Learn, please pay attention to the YAML pipeline default behaviors:

    • Since there are no explicit dependencies set for stage Terraform_Apply_${{ parameters.environment }}, by default, it depends on the stage just before it in the YAML file.
    • Since there are no explicit conditions set for stage Terraform_Apply_${{ parameters.environment }}, it will use the default condition, which means this stage runs if all direct and indirect dependencies have succeeded, as if you specified condition: succeeded().
    • In addition, when you specify your own condition property for a stage / job / step, you overwrite its default condition: succeeded(). Therefore, you may consider adding the same condition in stage Terraform_Apply_${{ parameters.environment }} as that you defined for stage Any_Tf_Changes_${{ parameters.environment }}.
    • Furthermore, please also make sure you take into account the state of the parent stage Any_Tf_Changes_${{ parameters.environment }} when writing your own conditions, since overwriting the default condition can lead to your stage running even if the build is cancelled or failed.

    With those being said, if your actual expectation is to run the stage Terraform_Apply_${{ parameters.environment }} only when the stage Any_Tf_Changes_${{ parameters.environment }} is succeeded, you can try adding the expression eq(dependencies.Any_Tf_Changes_${{ parameters.environment }}.result, 'Succeeded') or succeeded('Any_Tf_Changes_${{ parameters.environment }}') for apply stage. Here is the part of stage Terraform_Apply_${{ parameters.environment }} of the YAML pipeline for your reference.

    - stage: Terraform_Apply_${{ parameters.environment }}
  displayName: Apply ${{ parameters.environment }}
  condition: and(
      eq('true', 'true'),
      eq(dependencies.Terraform_Plan_${{ parameters.environment }}.outputs['Terraform_Plan_${{ parameters.environment }}.anyTfChanges.anyTfChanges'], 'true'),
      succeeded('Any_Tf_Changes_${{ parameters.environment }}') 
    )
  variables:
    anyTfChanges: $[ stageDependencies.Terraform_Plan_${{ parameters.environment }}.Terraform_Plan_${{ parameters.environment }}.outputs['anyTfChanges.anyTfChanges'] ]
    deps: $[convertToJson(dependencies)]
    stageDeps: $[convertToJson(stageDependencies)]
  jobs:
  - deployment: Apply
    environment: ${{ parameters.environmentDisplayName }}
    displayName: Apply Terraform
    # pool: 'selfhosted'
    strategy:
      runOnce:
        deploy:
          steps:
          - checkout: self
          - powershell: |
              Write-Host "hello world"
              Write-Host "$(anyTfChanges)"
              Write-Host "deps is $(deps)"
              Write-Host "stageDeps is $(stageDeps)"

    enter image description here
    enter image description here

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search