skip to Main Content

I’ve got a bash script in a GitHub action that greps for image tags, then replaces them in a docker-compose.yml.

# docker-compose.yml from a JSON payload
docker_compose='"services:n  api:n    proxy: ghcr.io/org/api:main-4095094301n    ports:n      - 0:80n    restart: alwaysnn  proxy:n    image: ghcr.io/org/proxy:main-4095124301n    depends_on:n      - apin    environment:n      - "ENVIRONMENT=dev"n      - "PORT=8000"n    ports:n      - 0:3000n    restart: alwaysnn ..."'

# Extract the tag of each image, send to GitHub actions output
echo "proxy_tag=$(echo -n $docker_compose | grep -oP 'proxy:K[-w]+')" >> $GITHUB_OUTPUT

If I remove the >> operator to echo the output, everything looks OK:

proxy_tag=main-4095094301

But when I feed it into sed later in a different step of the pipeline, an extra newline character seems to come from nowhere:

echo "running s/PROXY_TAG/$proxy_tag/"
sed -i "s/PROXY_TAG/$proxy_tag/" docker-compose.yml

# running s/PROXY_TAG/main-4095094301
# /
# sed: -e expression #1, char 18: unterminated `s' command
# Error: Process completed with exit code 1.

I’ve tried some common suggestions eg. piping output through tr -d 'n':

echo "proxy_tag=$(echo -n $docker_compose | grep -oP 'proxy:K[-w]+' | tr -d 'n')" >> $GITHUB_OUTPUT

Is there something missing I don’t understand about bash vars or Github actions?

See below a more complete context of how where these commands are being used.

name: Update Docker Compose

on:
  workflow_dispatch:
    inputs:
      proxy_tag:
        description: proxy_tag
        type: string
        required: false
      # api_tag:
      # site_tag:

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Fetch docker-compose
        id: get-tags
        run: |
            docker_compose=$(curl ...)

            echo "proxy_tag=$(echo -n $docker_compose | grep -oP 'proxy:K[-w]+')" >> $GITHUB_OUTPUT
            echo "api_tag=$(echo $docker_compose | grep -oP 'api:K[-w]+')" >> $GITHUB_OUTPUT
            echo "site_tag=$(echo $docker_compose | grep -oP 'site:K[-w]+')" >> $GITHUB_OUTPUT

      - name: Replace tags docker-compose
        env:
          proxy_tag: >
            ${{ github.event_name == 'workflow_dispatch'
                && github.event.inputs.proxy_tag
              || steps.get-tags.outputs.proxy_tag
              || 'latest'
            }}
          # api_tag: >
          # site_tag: >
        run: |
          echo "Setting [$proxy_tag]"

          sed -i "s/PROXY_TAG/$proxy_tag/" docker-compose.yml
          sed -i "s/API_TAG/$api_tag/" docker-compose.yml
          sed -i "s/SITE_TAG/$site_tag/" docker-compose.yml

        # Outputs:
        #   Setting [main-4095094301       # <--- notice the newline
        #   ]
        #
        #   sed: -e expression #1, char 18: unterminated `s' command
        #   Error: Process completed with exit code 1.

2

Answers


  1. I suspect that the newline character being added is from the YAML’s folding using > while setting your environment variables under env. Try >- to not add (strip) the last newline character.

    This (https://yaml-multiline.info/) might be helpful for live experimentation with YAML.

    Login or Signup to reply.
  2. Worst case, if you’re using bash,

    $: proxy_tag="main-4095094301
    " # embedded newline for example
    
    $: echo "[$proxy_tag]" # show with embedded newline
    [main-4095094301
    ]
    
    $: echo "[${proxy_tag%$'n'}]" # show with embedded newline removed
    [main-4095094301]
    
    $: echo PROXY_TAG | sed "s/PROXY_TAG/$proxy_tag/" # syntax error with embedded newline
    sed: -e expression #1, char 27: unterminated `s' command
    
    $: echo PROXY_TAG | sed "s/PROXY_TAG/${proxy_tag%$'n'}/" # removed, syntax ok
    main-4095094301
    

    That seems error-prone to me, though, and I prefer to do most of my basic string processing right in the interpreter unless it’s huge and I need it faster, so I’d probably do something based on this –

    $: docker_compose=$'services:n  api:n    proxy: ghcr.io/org/api:main-4095094301n    ports:n      - 0:80n    restart: alwaysnn  proxy:n    image: ghcr.io/org/proxy:main-4095124301n    depends_on:n      - apin    environment:n      - "ENVIRONMENT=dev"n      - "PORT=8000"n    ports:n      - 0:3000n    restart: alwaysnn ...' # note $'...' for clean quotes and newlines
    
    $: echo "[$docker_compose]"
    [services:
      api:
        proxy: ghcr.io/org/api:main-4095094301
        ports:
          - 0:80
        restart: always
    
      proxy:
        image: ghcr.io/org/proxy:main-4095124301
        depends_on:
          - api
        environment:
          - "ENVIRONMENT=dev"
          - "PORT=8000"
        ports:
          - 0:3000
        restart: always
    
     ...]
    
    $: shopt -s extglob                            # enable extended globbing
    $: tmp="${docker_compose##* proxy: *([^:]):}"  # trim front - *(...) is "0 or more"
    $: proxy_tag="${tmp%%$'n'*}"                  # trim after, from 1st newline on
    $: echo "[$proxy_tag]"                         # show that it's clean
    [main-4095094301]
    
    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search