Json Question posted in
Our archive of expertly curated questions and answers provides insights and solutions to common problems related to this popular data interchange format. From parsing and manipulating JSON data to integrating it with various programming languages and web services, our archive has got you covered. Start exploring today and take your JSON skills to the next level

Convert Flat json to tree with Nested array

I’m trying convert JSON by NiFi JOLTTransformRecord processor to tree with nested array.

But I am having some trouble with converting the flat JSON to Nested JSON. I have looked at examples and didn’t get any closer as to what is mentioned above. I need to transform a JSON structure by using a JOLT spec.
What I’m doing wrong?
Help Me please/

Source JSON

[
  {
    "log_date": "2024-07-24 16:10:22.851",
    "log_level": "INFO",
    "log_message": "Message1",
    "request_id": "0a3d546d"
  },
  {
    "log_date": "2024-07-24 16:10:22.851",
    "log_level": "INFO",
    "log_message": "Message2",
    "request_id": "0a3d546d"
  },
  {
    "log_date": "2024-07-24 16:10:22.851",
    "log_level": "INFO",
    "log_message": "Message3",
    "request_id": "0a3d546d"
  },
  {
    "log_date": "2024-07-24 16:10:22.572",
    "log_level": "INFO",
    "log_message": "Message4",
    "request_id": "0a3d546d"
  },
  {
    "log_date": "2024-07-24 16:10:22.572",
    "log_level": "INFO",
    "log_message": "Message5",
    "request_id": "0a3d546d"
  },
  {
    "log_date": "2024-07-24 16:10:22.572",
    "log_level": "INFO",
    "log_message": "Message6",
    "request_id": "0a3d546d"
  }
]

DESIRABLE output

{
  "request_id": "0a3d546d",
  "log": [
    {
      "log_date": "2024-07-24 16:10:22.851",
      "log_level": "INFO",
      "log_message": "Message1"
    },
    {
      "log_date": "2024-07-24 16:10:22.851",
      "log_level": "INFO",
      "log_message": "Message2"
    },
    {
      "log_date": "2024-07-24 16:10:22.851",
      "log_level": "INFO",
      "log_message": "Message3"
    },
    {
      "log_date": "2024-07-24 16:10:22.572",
      "log_level": "INFO",
      "log_message": "Message4"
    },
    {
      "log_date": "2024-07-24 16:10:22.572",
      "log_level": "INFO",
      "log_message": "Message5"
    },
    {
      "log_date": "2024-07-24 16:10:22.572",
      "log_level": "INFO",
      "log_message": "Message6"
    }
  ]
}

I’m trying use this specification

JOLT Specification

[
  {
    "operation": "shift",
    "spec": {
      "*": {
        "request_id": "request_id",
        "log*": {
          "@(1)": "log"
        }
      }
    }
  },
  {
    "operation": "cardinality",
    "spec": {
      "request_id": "ONE"
    }
  },
  {
    "operation": "cardinality",
    "spec": {
      "logs": {
        "log_date;log_level;log_message": "ONE"
      }
    }
  }
]
Tags:

2

Answers


  1. 0

    You can make it online here.
    You can use the Java library from here.
    Also if only the given above JOLT specification would be used, you can write code using only the JSON library (Python json package, for example).

    Login or Signup to reply.
  2. 0

    You might prefer using a single shift transformation spec in which the indexes of the array are matched for the ones with zero vs. others such as

    [
  {
    "operation": "shift",
    "spec": {
      "0": {
        "request_id": "&",//only keep this value
        "*": "log[&1].&"//replicate these values arraywise with key "log"
      },
      "*": {//objects of the array with indexes other than zero
        "request_id": { "": "" },//match with empty matching in order to get rid of the extra "request_id" values
        "*": "log[&1].&"
      }
    }
  }
]
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search