Question 98209 in Nginx Question posted in
The official documentation can be found here.

How to give to the strategy matrix a dynamic object based on the current filenames in the folder – Nginx

my goal is to have some job steps (docker build and push) to run on every Dockerfile I add in a specific folder.

This is the main structure I’m trying:

...

stages:
- stage: Get_DF_Stage
  displayName: Get_Dockerfiles
  jobs:
    - job: Get_DF_Job
      steps:
        - task: bash@3
          inputs:
            filePath: 'helpers/from_files_to_yaml_object.sh'
          displayName: Get_Dockerfiles
          name: Get_DF_Task


# --- Take the latest image on the private repo and retags it with the current date
- stage: Docker_Image_Patch
  displayName: Docker_Image_Patch
  dependsOn:
    - Get_DF_Stage
  jobs:
  - job: docker_login
    displayName: Login to COMPANY CR

    steps:
    - task: Docker@2
      displayName: Login to COMPANY Container Registry
      inputs:
        command: login
        containerRegistry: COMPANY-cr
  
  - job: docker_patch
    displayName: Build and push Docker images

    strategy:
      # $[ stageDependencies.Get_DF_Stage.Get_DF_Job.outputs['Get_DF_Task.dockerfile_list'] ]
      matrix: 
        nginx:
          dockerfile: 'nginx'
        nginx-alpine:
          dockerfile: 'nginx-alpine'  

    steps:
    - task: Docker@2
      displayName: Build Docker image
      inputs:
        command: build
        containerRegistry: COMPANY-cr
        Dockerfile: dockerfiles/$(dockerfile)
        repository: $(dockerfile)
        buildContext: .
        arguments: --no-cache --pull
        tags: latest

    - task: Docker@2
      displayName: Push Docker image
      inputs:
        command: push
        containerRegistry: COMPANY-cr
        repository: $(dockerfile)
        tags: latest

My idea is to generate the YAML executing a bash task, that will have this output:

nginx:
  dockerfile: 'nginx'
nginx-alpine:
  dockerfile: 'nginx-alpine'

For two files in the folder, named: ‘nginx’ and ‘nginx-alpine’.

Then I would load that variable in the matrix through the stageDependencies command.

Is there a way to insert a YAML object from a bash script?
Or is there a simpler way to iterate jobs on files?

Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    In the end I've found out my solution wasn't working because I was trying to generate a YAML or a JSON Array, instead of a JSON MAP.

    So I've modified my bash task to generate a JSON map based on the files in a specific folder:

    #!/bin/bash
filenames_list="$(ls -1 dockerfiles/ |tr 'n' ' ' )"
filenames_list=($filenames_list) 

filenames_tmp=""
for (( i=0; i<${#filenames_list[@]}; i++ ))
do
    echo "> Found '${filenames_list[$i]}'"
    filenames_tmp="${filenames_tmp}, "${filenames_list[$i]}":{"dockerfile":"${filenames_list[$i]}"}"
done

# Remove first void comma
filenames_tmp="${filenames_tmp[0]:2}"
# Create JSON
filenames_list="{ ${filenames_tmp[*]} }"

## --- Output: { "nginx": { "dockerfile": "nginx" }, "nginx-alpine": { "dockerfile": "nginx-alpine" }}
echo "> Result <${filenames_list}>"
echo "##vso[task.setvariable variable=dockerfile_list;isOutput=true]`echo "${filenames_list}"`"

    Then I call the dockerfile variable from the second stage like this

    - stage: Docker_Image_Patch
  displayName: Docker_Image_Patch
  dependsOn:
    - Get_DF_Stage
  jobs:
  - job: docker_patch
  displayName: Build and push Docker images
  strategy:
    matrix: $[ stageDependencies.Get_DF_Stage.Get_DF_Job.outputs['Get_DF_Task.dockerfile_list'] ]

  steps:
    - pwsh: Write-Output "$(dockerfile)"
      displayName: Show current Dockerfile
    
    - script: echo "##vso[task.setvariable variable=CURRENT_DATE]`date +"%Y%m%d"`"

    - task: Docker@2
      displayName: Build Docker image
      inputs:
        command: build
        containerRegistry: company-cr
        Dockerfile: dockerfiles/$(dockerfile)
        repository: devops/$(dockerfile)
        buildContext: .
        arguments: --no-cache --pull
        tags: |
          $(CURRENT_DATE)
          latest

    - task: Docker@2
      displayName: Push Docker image
      inputs:
        command: push
        containerRegistry: company-cr
        repository: devops/$(dockerfile)
        tags: |
          $(CURRENT_DATE)
          latest

  2. 0

    Yes this is possible in following way:

    jobs:
- job: JobA
  steps:
  - pwsh: |
      $json="{'nginx': {'dockerfile': 'nginx'}, 'nginx-alpine': {'dockerfile': 'nginx-alpine'}}"
      Write-Host "##vso[task.setvariable variable=targets;isOutput=true]$json"
    name: setTargets
  - script: echo $(setTargets.targets)
    name: echovar

- job: buildSrc
  dependsOn: JobA
  displayName: Build source
  strategy:
    matrix: $[ dependencies.JobA.outputs['setTargets.targets'] ]
  variables:
    targets: $[ dependencies.JobA.outputs['setTargets.targets'] ]
  steps:
  - pwsh: Write-Host "${{ convertToJson(variables) }}"
    displayName: 'Print all variables via expression'

    You can also check my answer to similar question here

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search