Question 252462 in Json Question posted in
Our archive of expertly curated questions and answers provides insights and solutions to common problems related to this popular data interchange format. From parsing and manipulating JSON data to integrating it with various programming languages and web services, our archive has got you covered. Start exploring today and take your JSON skills to the next level

How to merge two JSON arrays into a single merged array using Jolt in NiFi

I have a requirement in which I need to merge two JSON arrays into one. Currently the JSON is like this:

[
  {
    "test": [
      {
        "count": 5,
        "name": "highway"
      }
    ]
  },
  {
    "test": [
      {
        "count": 6,
        "name": "plateau"
      }
    ]
  }
]

Now I need to merge the entire json array as a single array.

Like:

{
  "test": [
    {
      "count": 5,
      "name": "highway"
    },
    {
      "count": 6,
      "name": "plateau"
    }
  ]
}

I am new to Jolt and hence don’t know what to try

Tags:

2

Answers


  1. 0

    If you use Jackson and Lombok, you can deserialize the data and then reduce it.

    You should be able to do something similar to Jackson with Jolt. The underlying reduce is the main business logic here.

    import java.util.Arrays;

import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.JsonMappingException;
import com.fasterxml.jackson.databind.ObjectMapper;

import lombok.AllArgsConstructor;
import lombok.NoArgsConstructor;
import lombok.Data;

public class MergeData {
    public static final String DATA = "[{"test":[{"count":5,"name":"highway"}]},{"test":[{"count":6,"name":"plateau"}]}]";
    private static final ObjectMapper MAPPER = new ObjectMapper();

    public static void main(String[] args) throws JsonMappingException, JsonProcessingException {
        TestModel[] testModels = MAPPER.readValue(DATA, TestModel[].class);
        System.out.println(Arrays.toString(testModels));

        TestModel reduced = reduceTestModels(testModels);
        System.out.println(reduced);
    }

    private static TestModel reduceTestModels(TestModel[] testModels) {
        TestObject[] testObjects = Arrays.stream(testModels)
                .map(TestModel::getTest)
                .flatMap(Arrays::stream)
                .toArray(TestObject[]::new);

        return new TestModel(testObjects);
    }

    @Data
    @NoArgsConstructor
    @AllArgsConstructor
    private static class TestModel {
        private TestObject[] test;

        @Override
        public String toString() {
            return String.format("{ "test": %s }", Arrays.toString(test));
        }
    }

    @Data
    @NoArgsConstructor
    @AllArgsConstructor
    private static class TestObject {
        private int count;
        private String name;

        @Override
        public String toString() {
            return String.format("{ "count": %d, "name": "%s" }", count, name);
        }
    }
}

    Output:

    [
  { "test": [{ "count": 5, "name": "highway" }] },
  { "test": [{ "count": 6, "name": "plateau" }] }
]

    { "test": [{ "count": 5, "name": "highway" }, { "count": 6, "name": "plateau" }] }
    Login or Signup to reply.
  2. 0

    You can use the following shift transformation spec

    [
  {
    "operation": "shift",
    "spec": {
      "*": {
        "*": {
          "*": "&1[]" // &1 replicates the key "test" grabbing the literal's value 
                      // after going up the tree one level
        }
      }
    }
  }
]

    the demo on the site http://jolt-demo.appspot.com/ is :

    enter image description here

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search