Question 182127 in Json Question posted in
Our archive of expertly curated questions and answers provides insights and solutions to common problems related to this popular data interchange format. From parsing and manipulating JSON data to integrating it with various programming languages and web services, our archive has got you covered. Start exploring today and take your JSON skills to the next level

Convert json directly into dictionary of pandas dataframes

I have json files which look like a dictionary of a list of similar dictionaries:

{"People":[{"FirstName":"Max","Surname":"Smith"},{"FirstName":"Jane","Surname":"Smart"}],
"Animals":[{"Breed":"Cat","Name":"WhiteSocks"},{"Breed":"Dog","Name":"Zeus"}]}

I’m using the following code to convert this into a dictionary of pandas dataframes:

import pandas as pd
import json

# Read the json file
jsonFile = 'exampleJson.json'
with open(jsonFile) as j:
    data = json.load(j)

# Convert it to a dictionary of dataframes
dfDict = {}
for dfName, dfContents in data.items():
    dfDict[dfName] = pd.DataFrame(dfContents)
    display(dfDict[dfName])

The above code gives me exactly what I want, which is a dictionary of dataframes. However it seems rather inefficient. Is there a way to read the json directly into a dictionary of dataframes, rather than reading it into a json object first and then copying that into a dictionary of dataframes? The files I’m working with will be huge.

Tags:

2

Answers


  1. 0

    You should try this code:

    import pandas as pd
import json

# Read the json file
jsonFile = 'exampleJson.json'
with open(jsonFile) as j:
    data = pd.json_normalize(json.load(j))

# Convert it to a dictionary of dataframes
 df1=data['People']
 print(df1)
 df2=data['Animals']
 print(df2)
    Login or Signup to reply.
  2. 0

    You can use json_normalize():

    import json

import pandas as pd


json_file = "exampleJson.json"
with open(json_file) as j:
    data = json.load(j)

df_dict = {d: pd.json_normalize(data=data, record_path=d) for d in data}
print(df_dict)
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search