Question 178979 in Json Question posted in
Our archive of expertly curated questions and answers provides insights and solutions to common problems related to this popular data interchange format. From parsing and manipulating JSON data to integrating it with various programming languages and web services, our archive has got you covered. Start exploring today and take your JSON skills to the next level

How to use Jolt spec to transform data in Nifi?

I want this data to be transformed in a given way using the jolt spec of Nifi.

Conditions are if studentId and loc_id are the same then we will combine their info and will pass the rest as it is. There are empty fields as well "".

Data

[
  {
    "studentId": "2222",
    "loc_id": "L1",
    "topId": "Lotus",
    "SubID1": "A1",
    "SubID2": "B1"
  },
  {
    "studentId": "2222",
    "loc_id": "L1",
    "topId": "tulip",
    "SubID1": "A2",
    "SubID2": ""
  },
  {
    "studentId": "3333",
    "loc_id": "L3",
    "topId": "Rose",
    "SubID1": "A3",
    "SubID2": ""
  },
  {
    "studentId": "4444",
    "loc_id": "L3",
    "topId": "Rose",
    "SubID1": "A5",
    "SubID2": "B7"
  }
]

Data after jolt spec: First two data have same studentId and loc_id so we have combined their info and rest two have passed as it is with an addition of visit list

[
  {
    "studentId": "2222",
    "loc_id": "L1",
    "VisitList": [
      {
        "topId": "Lotus",
        "SubID1": "A1",
        "SubID2": "B1"
      },
      {
        "topId": "tulip",
        "SubID1": "A2",
        "SubID2": ""
      }
    ]
  },
  {
    "studentId": "3333",
    "loc_id": "L1",
    "VisitList": [
      {
        "topId": "Rose",
        "SubID1": "A3",
        "SubID2": ""
      }
    ]
  },
  {
    "providerId": "4444",
    "specillity": "L3",
    "VisitList": [
      {
        "topId": "Rose",
        "SubID1": "A5",
        "SubID2": "B7"
      }
    ]
  }
]
Tags:

2

Answers


  1. 0

    You can use the JoltTransformJSON processor. You’ll need to create a Jolt specification to combine the data based on studentId and loc_id.

    [
  {
    "operation": "shift",
    "spec": {
      "studentId": "studentId",
      "loc_id": "loc_id"
    }
  },
  {
    "operation": "group",
    "spec": {
      "studentId": "studentId",
      "loc_id": "loc_id",
      "VisitList": "*"
    }
  },
  {
    "operation": "merge",
    "spec": {
      "studentId": "studentId",
      "loc_id": "loc_id",
      "VisitList": "VisitList"
    }
  },
  {
    "operation": "shift",
    "spec": {
      "VisitList": "VisitList"
    }
  }
]
    Login or Signup to reply.
  2. 0

    You can use this spec:

    [
  {
    "operation": "shift",
    "spec": {
      "*": {
        "@": "@(1,studentId).@(1,loc_id)"
      }
    }
  },
  {
    "operation": "cardinality",
    "spec": {
      "*": {
        "*": "MANY"
      }
    }
  },
  {
    "operation": "shift",
    "spec": {
      "*": {
        "*": {
          "*": {
            "studentId": "[#4].&",
            "loc_id": "[#4].&",
            "topId|SubID*": "[#4].VisitList[&1].&"
          }
        }
      }
    }
  },
  {
    "operation": "cardinality",
    "spec": {
      "*": {
        "studentId": "ONE",
        "loc_id": "ONE"
      }
    }
  }
]

    1 – shift: You should create a unique root for every object. In this case, we create it by combining 2 values like this: @(1,studentId).@(1,loc_id).

    2 – cardinality: change every object that has 1 object to an array. We should do this to prepare our input for the next shift operation.

    3 – shift: create your desired output.

    4 – cardinality: change studentId and loc_id to a string in all objects.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search