Question 81371 in Docker Question posted in
The official documentation can be found here.

Override json file values with environment variables docker

Assume that I’ve a complex json file that is used to configurate my project.

Like the json below:

{
    "apis": {
        "payment": {
            "base_url": "https://example.com/"
        },
        "order": {
            "base_url": "https://example.com/"
        },
        
    },
    "features": {
        "authentication": {
            "authProviders": true,
            "registration": false
        }
    },
    "availableLocales": [
      "en",
      "es"
    ]
}

With .Net there’s a feature that allows us to override the values based on environment variables.

If I wanted to override the value of apis.payment.base_url I could pass an environment variable: APIS__PAYMENT__BASE_URL and the value would be replaced.

Since I’m currently not using .Net is there any alternatives?
This is what I’m using right now, but this does not fit my needs

FROM code as prepare-build
ENV JQ_VERSION=1.6
RUN wget --no-check-certificate 
    https://github.com/stedolan/jq/releases/download/jq-${JQ_VERSION}/jq-linux64 
    -O /tmp/jq-linux64
RUN cp /tmp/jq-linux64 /usr/bin/jq
RUN chmod +x /usr/bin/jq
WORKDIR /code/public
RUN jq 'reduce path(recurse | scalars) as $p (.;setpath($p; "$" + ($p | join("_"))))'  
    ./configurations/settings.json > ./configurations/settings.temp.json && 
    yez | cp ./configurations/settings.temp.json ./configurations/settings.json
WORKDIR /code/deploy
RUN echo "#!/usr/bin/env sh" | tee -a /code/deploy/start.sh > /dev/null && 
    echo 'export EXISTING_VARS=$(printenv | awk -F= '''{print $1}''' | sed '''s/^/$/g''' | paste -sd,);' | tee -a /code/deploy/start.sh > /dev/null && 
    echo 'for file in $CONFIGURATIONS_FOLDER;' | tee -a /code/deploy/start.sh > /dev/null && 
    echo 'do' | tee -a /code/deploy/start.sh > /dev/null && 
    echo '   cat $file | envsubst $EXISTING_VARS | tee $file' | tee -a /code/deploy/start.sh > /dev/null && 
    echo 'done' | tee -a /code/deploy/start.sh > /dev/null && 
    echo 'nginx -g '''daemon off;'''' | tee -a /code/deploy/start.sh > /dev/null
WORKDIR /code

This was I have a problem that, I need to pass all the json paths as environment variables, to override it correctly. If not, the variables will be replaced with the path of it, only.

I think the best approach would be:

Read the environment variables and create a json file with their values, then override the existing json file with the values of the created one.

Does anyone have any thing that could help me achieve this?

To summarize.

In order to make easy to identify which environment variables I should use, let’s assume it will have a prefix of SETTINGS.
Example of how I would override values.

JSON PATH EQUIVALENT ENVIRONMENT VARIABLE
APIS.PAYMENT.BASE_URL SETTINGS__APIS__PAYMENT__BASE_URL
AVAILABLELOCALES[0] SETTINGS__AVAILABLELOCALES__0
Tags:

2

Answers


  1. 0

    I’m a jq novice, and I’d be very interested in a better jq script, but here’s one way to use environment variables to modify a settings.json file.

    $ cat settings.json 
{
  "apis": {
    "payment": {
      "base_url": "https://example.com/"
    },
    "order": {
      "base_url": "https://example.com/"
    }
  },
  "features": {
    "authentication": {
      "authProviders": true,
      "registration": false
    }
  },
  "availableLocales": [
    "en",
    "es"
  ]
}

$ printenv|grep SETTINGS__
SETTINGS__APIS__PAYMENT__BASE_URL=https://example2.com
SETTINGS__AVAILABLELOCALES__0=cs

$ jq -n '
inputs as $i
| [ $i
    | ..
    | keys_unsorted?
    | .[]
    | strings
  ]
| unique as $allKeys
|
def fixCase:
  . as $w
  | reduce ($allKeys[]|select(length == ($w|length))) as $k
      ("";. + $k|match($w;"i").string)
;
def envpaths:
  [
    $ENV
    | to_entries[]
    | select(.key | startswith("SETTINGS__"))
    | [[ (.key|split("__"))[1:][]
         | if test("^[0-9]+$") then tonumber else fixCase end
       ],
         .value
      ]
  ]
;
reduce envpaths[] as $p ($i; .|setpath($p[0];$p[1]))' settings.json
# the output
{
  "apis": {
    "payment": {
      "base_url": "https://example2.com"
    },
    "order": {
      "base_url": "https://example.com/"
    }
  },
  "features": {
    "authentication": {
      "authProviders": true,
      "registration": false
    }
  },
  "availableLocales": [
    "cs",
    "es"
  ]
}

    See it work on jqplay.org.

    Login or Signup to reply.
  2. 0

    The task can be solved using jq.
    The version is robust against settings that do not match a path in the document.

    Variables

    SETTINGS__APIS__PAYMENT__BASE_URL=https://example2.com
SETTINGS__AVAILABLELOCALES__0=cs
SETTINGS__UNAVAILABLE__PATH=1

    Code

    jq 'def settings:
      def prepareVariables:
        [$ENV | to_entries[] | select(.key | startswith("SETTINGS__"))]   # select all variables that starts with "SETTINGS__"
        | map(.key |= (. / "__" | map(tonumber? // .))[1:]);              # convert variable names to path arrays

      [paths(scalars) | [., map(ascii_upcase? // .)]] |                   # collect all leaf paths from input file and add uppercase path
      reduce .[] as $leafPath                                             # add leaf paths to corresponding settings
             (prepareVariables; map(select($leafPath[1] == .key) |= . + {path: $leafPath[0]})) |
      map(select(has("path")));                                           # drop settings for unknown paths

    . as $input |
    reduce settings[] as $setting                                         # apply new settings from variables to input file
           ($input; . | setpath($setting["path"]; $setting["value"]))
' input.json

    Output

    {
  "apis": {
    "payment": {
      "base_url": "https://example2.com"
    },
    "order": {
      "base_url": "https://example.com/"
    }
  },
  "features": {
    "authentication": {
      "authProviders": true,
      "registration": false
    }
  },
  "availableLocales": [
    "cs",
    "es"
  ]
}
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search