I have valious formats of incoming JSON from 3rd parties that I need to process in unified manner. Considering the data structure is not normalized, I could not apply unified logic of data processing. To achieve that I’m seeking for data formatting into single format before processing.
Source 1:
{
"fieldName": "status",
"oldValue": "Open",
"newValue": "Closed"
}
Source 2:
{
"fieldName": "status",
"oldValue": {
"name": "Open"
},
"newValue": {
"name": "Closed"
}
}
Source 3:
{
"fieldName": "reason",
"oldValue": null,
"newValue": {
"name": "Requirements changed"
}
}
Source 4:
{
"fieldName": "advertise",
"oldValue": true,
"newValue": false
}
In result transformation I would like to have the following (combined for all sources):
{
"status"|"reason"|"advertise": {
"oldValue": {
"value": "Open"|null|true
},
"newValue": {
"value": "Closed"|"Requirements changed"|false
}
}
}
I’m struggling with various combinations of shift and default but could not achieve the transformation. I tried to build JOLT specification that makes this transformation (see example below), however I’m unable to put different field structure (simple value vs object) into unified format.
[
{
"operation": "shift",
"spec": {
"fieldName": {
"*": {
"@(2,oldValue)": "&1.oldValue",
"@(2,newValue)": "&1.newValue"
}
}
}
}
]
Sample of code (Java) to reproduce the problem
package org.example;
import com.bazaarvoice.jolt.Chainr;
import com.bazaarvoice.jolt.JsonUtils;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.util.List;
public class Main {
public static void main(String[] args) throws IOException {
var spec = read("src/main/resources/spec.json");
var chainr = Chainr.fromSpec(JsonUtils.jsonToList(spec));
var example1 = read("src/main/resources/example1.json");
var example2 = read("src/main/resources/example2.json");
var example3 = read("src/main/resources/example3.json");
var example4 = read("src/main/resources/example4.json");
for (String source: List.of(example1, example2, example3, example4)) {
System.out.println("===nSource n" + source);
var transformedSource = chainr.transform(JsonUtils.jsonToMap(source));
System.out.println("nAfter transformation n" +
JsonUtils.toPrettyJsonString(transformedSource));
}
}
private static String read(String path) throws IOException {
return new String(Files.readAllBytes(Paths.get(path)));
}
}
You also need the following dependencies to achieve data transformation
<dependency>
<groupId>com.bazaarvoice.jolt</groupId>
<artifactId>jolt-core</artifactId>
<version>0.1.8</version>
</dependency>
<dependency>
<groupId>com.bazaarvoice.jolt</groupId>
<artifactId>json-utils</artifactId>
<version>0.1.8</version>
</dependency>
Any thoughts how to make it possible?
Thanks in advance!
2
Answers
The following spec should work for all the scenarios:
More straightforward option would be