Question 297135 in Json Question posted in
Our archive of expertly curated questions and answers provides insights and solutions to common problems related to this popular data interchange format. From parsing and manipulating JSON data to integrating it with various programming languages and web services, our archive has got you covered. Start exploring today and take your JSON skills to the next level

Removing NaNs in a JSON file using python

Json file example:

{
  "name": "John Doe",
  "age": 30,
  "height": null,
  "weight": NaN,
}
{
  "name": "Jim Hanks",
  "age": NaN,
  "height": NaN,
  "weight": NaN,
}

now imagine there are a lot of rows some containing only NaNs some containing a few NaNs.. and so on

For every row NaNs have to removed not gsubbed or anything like that

I tried approaches like using import math and isnan() but my function only gsubbed it with null.
I`m not really a python guy :/

output should be:

{
"name": "John Doe",
"age": 30,
"height": null,
}
{
"name": "Jim Hanks",
}

all help is appreciated

Tags:

2

Answers


  1. 0

    What about using external function to process the json file?

    import json

def remove_nans(obj):
    if isinstance(obj, dict):
        # Recursively remove NaNs from dictionary
        return {key: remove_nans(value) for key, value in obj.items() if not (isinstance(value, float) and math.isnan(value))}
    elif isinstance(obj, list):
        # Recursively remove NaNs from list
        return [remove_nans(item) for item in obj]
    else:
        return obj

# with open('data.json', 'r') as f:
#    data = json.load(f)

data = [
    {
        "name": "John Doe",
        "age": 30,
        "height": None,
        "weight": float('nan')
    },
    {
        "name": "Jim Hanks",
        "age": float('nan'),
        "height": float('nan'),
        "weight": float('nan')
    }
]

processed_data = [remove_nans(row) for row in data]
    Login or Signup to reply.
  2. 0

    Can help

    processed_data = [remove_nans(row) for row in data]

def remove_nans(obj):
    return {key: value for key, value in obj.items() if value is not NaN and not (isinstance(value, float) and math.isnan(value))}
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search