Question 108840 in Json Question posted in
Our archive of expertly curated questions and answers provides insights and solutions to common problems related to this popular data interchange format. From parsing and manipulating JSON data to integrating it with various programming languages and web services, our archive has got you covered. Start exploring today and take your JSON skills to the next level

Removing elements from JSON payload with JOLT transformation

I’m rather new to JOLT transformations so I’m still having trouble with pretty basic stuff.

I have a big JSON payload that I need to simplify by removing most data from it.
The only attributes I want to keep are a normal attribute at the root level:

"root_element" : 14,

and a specific field from array elements:

"array_elements": [
  {
    "array_field": 103854
  },
  {
    "array_field": 9975
  },
  {
    "array_field": 47551
  }
],

It’s impractical to use the "remove" operation since there are a large number of attributes at both root level and within array_elements.

If I use this JOLT transformation:

{
    "operation": "shift",
    "spec": {
      "array_elements": {
        "*": {
          "array_field": "[&1].array_field"
        }
      }
    }
 }

I get the following result:

[
  {
    "array_field": 103854
  },
  {
    "array_field": 9975
  },
  {
    "array_field": 47551
  }
]

Then I try to add the required root attribute to my transformation:

{
    "operation": "shift",
    "spec": {
    "root_element" : "root_element",
      "array_elements": {
        "*": {
          "array_field": "[&1].array_field"
        }
      }
    }
 }

But this only returns this:

{
  "root_element" : 14
}

How can I modify my JOLT transformation in order to generate this JSON?

{
 "root_element" : 14,
"array_elements":[
  {
    "array_field": 103854
  },
  {
    "array_field": 9975
  },
  {
    "array_field": 47551
  }
 ] 
}

there are more attributes in-between for my real case. As an Example:

{
  "root_element": 14,
  "another_root_element": "text",
  "array_elements": [
    {
      "array_field": 103854,
      "another_array_field": "more text"
    },
    {
      "array_field": 9975
    },
    {
      "array_field": 47551
    }
  ]
}

I need to remove the "another" elements but imagine there are multiple "another" elements (hundreds) with various names (too many and random). I just need to keep the attributes I described.

Thanks in advance!

Tags:

2

Answers


  1. 0

    You are trying to add array_field items to an array. I said that because you have written [&1].

    So for root_element you want to add it to an object with the root_element key.

    Now suppose, How JOLT can see that?

    You write root_element as the first key you want to get in your JOLT spec, So JOLT creates an object for you and put root_element into that.

    But for the next key (array_elements) you want to put that into an array but you have an object currently. So jolt ignore that key and create an object with just the root_element key in desired output for you.

    For fixing that you can for example put array_elements in an object named arr, So you can have the following JOLT spec:

    [
  {
    "operation": "shift",
    "spec": {
      "root_element": "root_element",
      "array_elements": {
        "*": {
          "array_field": "arr[&1].array_field"
        }
      }
    }
 }
]

    Output:

    {
  "root_element": 14,
  "arr": [
    {
      "array_field": 103854
    },
    {
      "array_field": 9975
    },
    {
      "array_field": 47551
    }
  ]
}
    Login or Signup to reply.
  2. 0

    You can use the following transformation considering those other attributes :

    [
  {
    "operation": "shift",
    "spec": {
      "root_element": "&", // pick attribute named "root_element" only
      "array_elements": {
        "*": {
          // pick attributes named "array_field" only
          "array_field": "&2[&1].&" // &2 grabs value after going two levels up the tree
                                    // &  replicates the current key-value combination
        }
      }
    }
  }
]

    the combination "&2[&1].&" might also be replaced with "&2[#2].&" as anternative to suppress the null generation for same cases you may encounter.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search