skip to Main Content

Say, I want to dynamically edit a Kubernetes deployment file that looks like this using Python:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: frontend
spec:
  replicas: 4
  selector:
    matchLabels:
      app: guestbook
      tier: frontend
  template:
    metadata:
      labels:
        app: guestbook
        tier: frontend
    spec:
      containers:
      - env:
        - name: GET_HOSTS_FROM
          value: dns
        image: gcr.io/google-samples/gb-frontend:v4
        name: php-redis
        ports:
        - containerPort: 80
        resources:
          requests:
            cpu: 100m
            memory: 100Mi

I have a code that opens this yaml file where I want to change the content of spec.replicas branch from 2 to 4:

 with open(deployment_yaml_full_path, "r") as stream:
        try:
            deployment = yaml.safe_load(stream)
            if value_to_change[0] == 'spec.replicas':
                deployment['spec']['replicas'] = value_to_change[1]
        except yaml.YAMLError as exc:
            logger.error('There was a problem opening a deployment file in path: ',
                         deployment_yaml_full_path=deployment_yaml_full_path, exc=exc)

I would like to know if there’s a way to avoid the hardcoded part here to something more dynamic:

if value_to_change[0] == 'spec.replicas':
                deployment['spec']['replicas'] = value_to_change[1]

Is there a way?

4

Answers


  1. target = value_to_change[0].split('.')
    
    if len(target) == 2 and target[0] in deployment and target[1] in deployment[target[0]]:
        deployment[target[0]][target[1]] = value_to_change[1]
    

    If the paths can be longer:

    path_len = len(target)
    
    d = deployment
    path_exists = True
    for i in range(path_len):
      if target[i] in d:
        d = d[target[i]]
      else:
        path_exists = False
    
    if path_exists:
       d = value_to_change[1]
    
    Login or Signup to reply.
  2. I believe you want to change the YAML to JSON/dictionary using PyYaml

    import yaml
    import json
    
    with open('config.yml', 'r') as file:
        configuration = yaml.safe_load(file)
    
    with open('config.json', 'w') as json_file:
        json.dump(configuration, json_file)
        
    output = json.dumps(json.load(open('config.json')), indent=2)
    print(output)
    

    After that you would like to use:

    class obj(object):
    def __init__(self, d):
        for k, v in d.items():
            if isinstance(k, (list, tuple)):
                setattr(self, k, [obj(x) if isinstance(x, dict) else x for x in v])
            else:
                setattr(self, k, obj(v) if isinstance(v, dict) else v)
    

    Usage Example:

    >>> d = {'a': 1, 'b': {'c': 2}, 'd': ["hi", {'foo': "bar"}]}
    >>> x = obj(d)
    >>> x.b.c
    2
    >>> x.d[1].foo
    'bar'
    

    The last phase will be to change the value by string path:

    from collections.abc import MutableMapping
    
    def set_value_at_path(obj, path, value):
        *parts, last = path.split('.')
    
        for part in parts:
            if isinstance(obj, MutableMapping):
                obj = obj[part]
            else:
                obj = obj[int(part)]
    
        if isinstance(obj, MutableMapping):
            obj[last] = value
        else:
            obj[int(last)] = value
    
    Login or Signup to reply.
  3. Disclaimer: this is solely for solace, fun and educational purpose

    I think the correct way to do what you want is to study json path, there is an easy example here and this stupid answer of mine could help you create the actual json path expressions!

    Well, this is the one-liner you should NOT use to achieve what you want in the most dynamic way possible:

    exec(f'deployment{"".join([f"[##{val}##]" for val in value_to_change[0].split(".")])}={value_to_change[1]}'.replace('##','"'))
    

    We create a list from the value_to_change[0] value splitting by dot.

      value_to_change[0].split(".")
    

    We get each val in this list end we enclose it in "dictionary" syntax (hashtags.. well they can be anything you want, but f-strings do not support backslashes, hence I replace hashtags with the quotes after)

    [f"[##{val}##]" for val in value_to_change[0].split(".")]
    

    We join the vals in a string and replace the hashtags and add the deployment string

    f'deployment{"".join([f"[##{val}##]" for val in value_to_change[0].split(".")])}={value_to_change[1]}'.replace('##','"')
    

    The result will be this string… (what you would normally write to change that value):

    'deployment["spec"]["replicas"]=50'
    

    We perform actual monstrosity executing the string.
    In my example value_to_change[1] is 50

    {
      "apiVersion": "apps/v1",
      "kind": "Deployment",
      "metadata": {
        "name": "frontend"
      },
      "spec": {
        "replicas": 50,
        "selector": {
          "matchLabels": {
            "app": "guestbook",
            "tier": "frontend"
          }
        },
        "template": {
          "metadata": {
            "labels": {
              "app": "guestbook",
              "tier": "frontend"
            }
          },
          "spec": {
            "containers": [
              {
                "env": [
                  {
                    "name": "GET_HOSTS_FROM",
                    "value": "dns"
                  }
                ],
                "image": "gcr.io/google-samples/gb-frontend:v4",
                "name": "php-redis",
                "ports": [
                  {
                    "containerPort": 80
                  }
                ],
                "resources": {
                  "requests": {
                    "cpu": "100m",
                    "memory": "100Mi"
                  }
                }
              }
            ]
          }
        }
      }
    }
    

    Have FUN!

    Login or Signup to reply.
  4. The json path solution is like this

    You just need parse function from jsonpath_ng library (pip install jsonpath_ng)

    from jsonpath_ng import parse
    
    # create the expression
    expr = parse(f'$.{".".join(value_to_change[0].split("."))}')
    # you can check if it works with expr.find(deployment) more after.
    
    # actually change the value
    expr.update(deployment, value_to_change[1])
    
    # deployment['spec']['replicas'] now has changed
    

    When you check with find you will see a lot of output, just focus on the first part:

    expr.find(deployment)
    >>> [DatumInContext(value=4, path=Fields('replicas')
    

    As you can see the expression is really easy: "$.path.to.value.you.want"
    If you wonder how to manage list inside, the syntax is the same as python [], with inside the index, or * to say all the items in the list (and this is priceless!!)

    This is a very good place to learn everything you need!

    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search