Question 272243 in Json Question posted in
Our archive of expertly curated questions and answers provides insights and solutions to common problems related to this popular data interchange format. From parsing and manipulating JSON data to integrating it with various programming languages and web services, our archive has got you covered. Start exploring today and take your JSON skills to the next level

Can't generate the valid .json file

I have the following python code, in order to generate a json code
It is reading the data from a csv file and the output will be written into a json file through dumps().

import os
import csv
import json

csvFilePath = "./mycsv.txt"
jsonFilePath = "./mydata.jsonc"

if os.path.exists(jsonFilePath):
    os.remove(jsonFilePath);

# Dictionary 1
myData1 = {
    "persons" : [
      {"Id": int,
      "groupId": str
      }
    ]
}

# Dictionary 2
myData2 = { 
    "data": [
      { "Id": int,
      "x": float,
      "y": float,
      "z": float
      }
    ]
}

# Init
myVarformyData1 = {}
myVarformyData2 = {}
myVarformyData1["persons"]={}
myVarformyData2["data"]={}

# Process first the sensor data (the first part persons)
with open(csvFilePath, 'r', encoding = 'utf-8') as csvFile:  
    line = csv.reader(csvFile)
    next(line)
    for line in csvFile:
        col = 0
        for value in line.split(';'):
            col = col + 1
            if (col == 3):
                myVarformyData1["persons"]["sId"] = value              
            elif (col == 8):
                myVarformyData1["persons"]["groupId"] = value
        
        with open(jsonFilePath, 'a', encoding = 'utf-8') as jsonFile1:
            jsonFile1.write(json.dumps(myVarformyData1, indent=4, sort_keys=True))
            jsonFile1.close();

# Closing the csv file
csvFile.close();

# Process other data (the second part data)
# Continues reading from the same .csv file 
with open(csvFilePath, 'r', encoding = 'utf-8') as csvFile:  
    line = csv.reader(csvFile)
    next(line)
    for line in csvFile:
        col = 0
        for value in line.split(';'):
            col = col + 1
            if (col == 3):
                myVarformyData2["data"]["sId"] = value
            elif (col == 10):
                myVarformyData2["data"]["x"] = value
            elif (col == 11):
                myVarformyData2["data"]["y"] = value
            elif (col == 12):
                myVarformyData2["data"]["z"] = value

        with open(jsonFilePath, 'a', encoding = 'utf-8') as jsonFile1:
            jsonFile1.write(json.dumps(myVarformyData2, indent=4,sort_keys=True))
            jsonFile1.close();

# Closing the .csv file
csvFile.close();

But my code above delivers the following .json file, which is not really correct:

{
    "persons": {
        "groupId": "1",
        "sId": "123"
    }
}{
    "persons": {
        "groupId": "2",
        "sId": "456"
    }
}{
    "persons": {
        "groupId": "3",
        "sId": "789"
    }
}{
    "data": {
        "sId": "123",
        "x": "6.11",
        "y": "2.188",
        "z": "1.6"
    }
}{
    "data": {
        "sId": "456",
        "x": "7",
        "y": "2.384",
        "z": "1.6"
    }
}{
    "data": {
        "sId": "789",
        "x": "8.74",
        "y": "1.159",
        "z": "1.6"
    }
}

What should I change in my data structure (dictionaries), in order to get the following output as a valid .json file:


{
    "persons": [
        {
        "groupId": "1",
        "sId": "123"
    },{
        "groupId": "2",
        "sId": "456"
    },{
        "groupId": "3",
        "sId": "789"
    }
], "data": [ 
    {
        "sId": "123",
        "x": "6.11",
        "y": "2.188",
        "z": "1.3"
    },{
        "sId": "456",
        "x": "7",
        "y": "2.384",
        "z": "1.4"
    },{
        "sId": "789",
        "x": "8.74",
        "y": "1.159",
        "z": "1.5"
    }
  ]
}
Tags:

2

Answers


  1. 0

    You are initializing myVarformyData1["persons"] value as a dict. To achieve the result you expected you need to use a list and append the data from each line to it as a dict:

    myVarformyData1 = {}
myVarformyData1["persons"] = [] # initialize persons field as a list

with open(csvFilePath, "r", encoding="utf-8") as csvFile:
    line = csv.reader(csvFile)
    next(line)
    for line in csvFile:
        person = {"sId": "", "groupId": ""} # initialize the dict for a single person
        col = 0
        for value in line.split(";"):
            col = col + 1
            if col == 3:
                person ["sId"] = value
            elif col == 8:
                person ["groupId"] = value

        myVarformyData1["persons"].append(person) # append person to the list

# this has to be outside the for loop, writing to the file a single time
with open(jsonFilePath, "a", encoding="utf-8") as jsonFile1:
    jsonFile1.write(
        json.dumps(myVarformyData1, indent=4, sort_keys=True)
    )
    jsonFile1.close()

    With this you solve the problem for the persons key. You can fix the rest of you code for the data key on a similar way.

    Login or Signup to reply.
  2. 0

    I believe you can simplify your code quite a bit. The key points here are using the CSV package to read your data and then assemble rows of output in lists that you can dump. I don’t have test data to work with, but I think this might do the trick.

    import csv
import json

csvFilePath = "./mycsv.txt"
jsonFilePath = "./mydata.jsonc"

persons, data = [], []
with open(csvFilePath, "r", encoding="utf-8") as csvFile:  
    for row in csv.reader(csvFile, delimiter=";"):
        persons.append({"sId": row[3], "groupId": row[8]})
        data.append({"sId": row[3], "x": row[10], "y": row[11], "z": row[12]})
results = {"persons": persons, "data": data}

with open(jsonFilePath, "w", encoding="utf-8") as jsonFile1:
    json.dump(results, jsonFile1, indent=4)
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search