Question 108910 in Json Question posted in
Our archive of expertly curated questions and answers provides insights and solutions to common problems related to this popular data interchange format. From parsing and manipulating JSON data to integrating it with various programming languages and web services, our archive has got you covered. Start exploring today and take your JSON skills to the next level

How to set data into a complex json dictionary in python?

Basically I need to set data into a JSON API Payload before I need to send it in the API request.
I have 2 files,

  1. text file which has JSON payload format, say json.txt
  2. yml file which has actual data, say tdata.yml
    Im writing a python code to create actual payload by inserting data from yml file to JSON payload format

Content of file 1

{
    "mpls_routing":
    {   
        "add_novel":
        {
            "device_ip": "DEVICE_IP",
            "discoveryHelloHoldTime": "DISCOVERYHELLOHOLDTIME",
            "discoveryHelloInterval": "DISCOVERYHELLOINTERVAL",
            "discoveryTargetedHelloHoldTime": "DISCOVERYTARGETEDHELLOHOLDTIME",
            "discoveryTargetedHelloInterval": "DISCOVERYTARGETEDHELLOINTERVAL",
            "downstreamMaxLabel": "DOWNSTREAMMAXLABEL",
            "downstreamMinLabel": "DOWNSTREAMMINLABEL",
            "initialBackoff": "INITIALBACKOFF",
            "instanceId": "554972419",
            "interfaceName": "INTERFACENAME",
            "isEntropyLabelEnabled": "ISENTROPYLABELENABLED",
            "isExplicitNullEnabled": "ISEXPLICITNULLENABLED",
            "isLoopDetectionEnabled": "ISLOOPDETECTIONENABLED",
            "isNsrEnabled": "ISNSRENABLED",
            "keepAliveInterval": "KEEPALIVEINTERVAL",
            "ldpId": "LDPID",
            "ldpIpMask": {
                "address": ""
        },
            "ldpPepSettings": [
                {
                    "com.novel.xmp.model.managed.standardTechnologies.ldp.LdpPepSettings": {
                        "labelDistributionMethod": "DISTMETHOD",
                        "ldpId": "LDPID",
                        "name": "MPLSINTERFACE",
                        "owningEntityId": "OWNINGENTITYID"
                    }
                }
            ],
            "maximumBackoff": "MAXIMUMBACKOFF",
            "sessionHoldTime": "SESSIONHOLDTIME"
        }
}

This is the yml data file

mpls_interface:
  DEVICE_IP: "10.104.120.141"
  DISCOVERYHELLOHOLDTIME: "15"
  DISCOVERYHELLOINTERVAL: "5"
  DISCOVERYTARGETEDHELLOHOLDTIME: "90"
  DISCOVERYTARGETEDHELLOINTERVAL: "10"
  DOWNSTREAMMAXLABEL: "289999"
  DOWNSTREAMMINLABEL: "24000"
  INITIALBACKOFF: "15"
  INTERFACENAME: "LOOPBACK0"
  ISENTROPYLABELENABLED: "FALSE"
  ISEXPLICITNULLENABLED: "FALSE"
  ISLOOPDETECTIONENABLED: "FALSE"
  ISNSRENABLED: "TRUE"
  KEEPALIVEINTERVAL: "60"
  LDPID: "192.168.0.141"
  MAXIMUMBACKOFF: "120"
  OWNINGENTITYID: ""
  SESSIONHOLDTIME: "180"
  DISTMETHOD: "LDP"
  MPLSINTERFACE: "TenGigE0/0/0/4"

I have following code which reads the JSON format file data

                template_data = file.read()
                template_data = json.loads(template_data)
                if "add_novel" in template_data:
                    full_template_file = template_data["add_novel"]
                    if testcase_name in full_template_file:
                        template_file_data = full_template_file[testcase_name]

And following code which reads the yml file

            with open(self.mpls_yml_file) as stream:
                yml_data = yaml.safe_load(stream)
                if "mpls_interface" in yml_data:
                    yml_file_data = yml_data["mpls_interface"]
            return yml_file_data

Then I pass both template_file_data and yml_file_data to following code to create the actual API payload.

testcase_data = {key: yml_file_data[value] for key, value in template_file_data.items() for k, v in
                             yml_file_data.items()}
            logger.info("n Testcase data %s" % testcase_data)

            return testcase_data

But I’m not able to get the right format since there is nested dictionary and also a array inside a dictionary.

Tags:

3

Answers


  1. 0

    The good news is that I don’t think you are planning to update the nested elements

    I am guessing this because the YAML file does not contain anything for ldpPepSettings, for example.

    So you could just use the original JSON information for parts where the YAML file does not provide an update. For example:

    testcase_data = {key: yml_file_data[value] for key, value in template_file_data.items() for k, v in
                             yml_file_data.items()}

    could become

    testcase_data = {
    key: 
    (yml_file_data[key] if key in yml_file_data else value) 
    for key, value in template_file_data.items()
}

    You will still have the problem that the two files have different keys. One has lowerCamelCase and the other has ALLCAPS.

    You could try to fudge it like this?

    testcase_data = {
    key: 
    (yml_file_data[key.upper()] if key.upper() in yml_file_data else value) 
    for key, value in template_file_data.items()
}
    Login or Signup to reply.
  2. 0

    The key here is to do the text search/replace to the JSON file’s contents before JSON decode it.

    #!/usr/bin/env python3
import json

import yaml

# Load the YAML with values
with open("values.yaml") as stream:
    lookups = yaml.safe_load(stream)

# Load the JSON as text
with open("payload.json") as stream:
    text = stream.read()

# Replace values
for lookup in lookups.values():
    new_text = text
    for old_value, new_value in lookup.items():
        new_text = new_text.replace(old_value, new_value)
    payload = json.loads(new_text)
    print(json.dumps(payload, indent=4))

    Output:

    {
    "mpls_routing": {
        "add_novel": {
            "device_ip": "10.104.120.141",
            "discoveryHelloHoldTime": "15",
            "discoveryHelloInterval": "5",
            "discoveryTargetedHelloHoldTime": "90",
            "discoveryTargetedHelloInterval": "10",
            "downstreamMaxLabel": "289999",
            "downstreamMinLabel": "24000",
            "initialBackoff": "15",
            "instanceId": "554972419",
            "interfaceName": "LOOPBACK0",
            "isEntropyLabelEnabled": "FALSE",
            "isExplicitNullEnabled": "FALSE",
            "isLoopDetectionEnabled": "FALSE",
            "isNsrEnabled": "TRUE",
            "keepAliveInterval": "60",
            "ldpId": "192.168.0.141",
            "ldpIpMask": {
                "address": ""
            },
            "ldpPepSettings": [
                {
                    "com.cisco.xmp.model.managed.standardTechnologies.ldp.LdpPepSettings": {
                        "labelDistributionMethod": "",
                        "ldpId": "192.168.0.141",
                        "name": "TenGigE0/0/0/4",
                        "owningEntityId": ""
                    }
                }
            ],
            "maximumBackoff": "120",
            "sessionHoldTime": "180"
        }
    }
}
    Login or Signup to reply.
  3. 0

    You need a recursive function that can navigate as deep as necessary in the JSON (Python dictionary) to substitute the values. Something like this:

    from yaml import safe_load
from json import load, dumps

YFILE = '/Volumes/G-Drive/mpls.yaml'
JFILE = '/Volumes/G-Drive/mpls.json'

def yaml_data(filename):
    with open(filename, 'r') as y:
        return safe_load(y)["mpls_interface"]

def json_data(filename):
    with open(filename, 'r') as j:
        return load(j)

def merge(d, y):
    if isinstance(d, dict):
        for k, v in d.items():
            if isinstance(v, dict):
                merge(v, y)
            elif isinstance(v, list):
                for e in v:
                    merge(e, y)
            else:
                d[k] = y.get(v, v)
        return d

result = merge(json_data(JFILE), yaml_data(YFILE))

print(dumps(result, indent=2))

    Output:

    {
  "mpls_routing": {
    "add_novel": {
      "device_ip": "10.104.120.141",
      "discoveryHelloHoldTime": "15",
      "discoveryHelloInterval": "5",
      "discoveryTargetedHelloHoldTime": "90",
      "discoveryTargetedHelloInterval": "10",
      "downstreamMaxLabel": "289999",
      "downstreamMinLabel": "24000",
      "initialBackoff": "15",
      "instanceId": "554972419",
      "interfaceName": "LOOPBACK0",
      "isEntropyLabelEnabled": "FALSE",
      "isExplicitNullEnabled": "FALSE",
      "isLoopDetectionEnabled": "FALSE",
      "isNsrEnabled": "TRUE",
      "keepAliveInterval": "60",
      "ldpId": "192.168.0.141",
      "ldpIpMask": {
        "address": ""
      },
      "ldpPepSettings": [
        {
          "com.novel.xmp.model.managed.standardTechnologies.ldp.LdpPepSettings": {
            "labelDistributionMethod": "LDP",
            "ldpId": "192.168.0.141",
            "name": "TenGigE0/0/0/4",
            "owningEntityId": ""
          }
        }
      ],
      "maximumBackoff": "120",
      "sessionHoldTime": "180"
    }
  }
}
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search