Question 110850 in Json Question posted in
Our archive of expertly curated questions and answers provides insights and solutions to common problems related to this popular data interchange format. From parsing and manipulating JSON data to integrating it with various programming languages and web services, our archive has got you covered. Start exploring today and take your JSON skills to the next level

Json – JQ – flatten out / unroll double-nested objects with different key names

Having the following json structure:

{
    "outerDescription1": {
        "innerDescription1": {
            "otherProperties": 1,
            "items": [
                "arrayItem1",
                "arrayItem2"
            ]
        }
    },
    "outerDescription2": {
        "innerDescription2": {
            "otherProperties": 2,
            "items": [
                "arrayItem3",
                "arrayItem4"
            ]
        }
    }
}

I would like to get the following result:

{
    "item": "arrayItem1",
    "outer": "outerDescription1",
    "inner": "innerDescription1",
    "otherProperties": 1
}
{
    "item": "arrayItem2",
    "outer": "outerDescription1",
    "inner": "innerDescription1",
    "otherProperties": 1
}
{
    "item": "arrayItem3",
    "outer": "outerDescription2",
    "inner": "innerDescription2",
    "otherProperties": 2
}
{
    "item": "arrayItem4",
    "outer": "outerDescription2",
    "inner": "innerDescription2",
    "otherProperties": 2
}

Assumption: there are many outerDescription and innerDescription keys and they are not known upfront.

Single level unroll is would be simple, but unrolling double-nested objects with different keys is a challenge to me.

The closest what I was able to get was:

jq "with_entries(.value = {outer: .key} + .value)[]"

which resulted in:

{                              
  "outer": "outerDescription1",
  "innerDescription1": {       
    "otherProperties": 1,      
    "items": [                 
      "arrayItem1",            
      "arrayItem2"             
    ]                          
  }                            
}                              
{                              
  "outer": "outerDescription2",
  "innerDescription2": {       
    "otherProperties": 2,      
    "items": [                 
      "arrayItem3",            
      "arrayItem4"             
    ]                          
  }                            
}

But right now, without knowing the next nested key name exactly, I am not able to unroll for a second time in the same way as outer would be swallowed.

I am using JQ 1.6

Tags:

3

Answers


  1. 0

    You could use path to find the according paths, and getpath to retrieve their values:

    path(.[][].items[]) as $p
| {item: getpath($p), outer: $p[0], inner: $p[1]}
+ (getpath($p[:-2]) | del(.items))

    {
  "item": "arrayItem1",
  "outer": "outerDescription1",
  "inner": "innerDescription1",
  "otherProperties": 1
}
{
  "item": "arrayItem2",
  "outer": "outerDescription1",
  "inner": "innerDescription1",
  "otherProperties": 1
}
{
  "item": "arrayItem3",
  "outer": "outerDescription2",
  "inner": "innerDescription2",
  "otherProperties": 2
}
{
  "item": "arrayItem4",
  "outer": "outerDescription2",
  "inner": "innerDescription2",
  "otherProperties": 2
}

    Demo

    Login or Signup to reply.
  2. 0

    If you know the name of your items property, the following equivalent programs are quite readable:

    to_entries[]
| { outer: .key, nested: (.value | to_entries[]) }
| { item: .nested.value.items[] }
+ { outer, inner: .nested.key }
+ (.nested.value | del(.items))

    to_entries[]
| { outer: .key, nested: (.value | to_entries[]) }
| { item: .nested.value.items[], outer, inner: .nested.key }
+ (.nested.value | del(.items))

    to_entries[]
| { outer: .key, nested: (.value | to_entries[]) }
| { outer }
+ (.nested
    | { inner: .key, item: .value.items[] }
    + (.value | del(.items))
)

    to_entries[]
| { outer: .key, nested: (.value | to_entries[]) }
| { outer }
+ (.nested
    | { inner: .key }
    + (.value | { item: .items[] } + del(.items))
)

    You could also use a function:

    def merge(name; field): { (name): field } + del(field);

to_entries[]
| { outer: .key, nested: (.value | to_entries[]) }
| { outer }
+ (.nested
    | { inner: .key }
    + (.value | merge("item"; .items[]))
)

    JSON does not care about key order in objects, but if – for whatever reason – you need to force a specific order, you append a final transformation. But this requires to list all keys again:

    ...
| { item, outer, inner, otherProperties } # force key/property order

    Output:

    {
  "item": "arrayItem1",
  "outer": "outerDescription1",
  "inner": "innerDescription1",
  "otherProperties": 1
}
{
  "item": "arrayItem2",
  "outer": "outerDescription1",
  "inner": "innerDescription1",
  "otherProperties": 1
}
{
  "item": "arrayItem3",
  "outer": "outerDescription2",
  "inner": "innerDescription2",
  "otherProperties": 2
}
{
  "item": "arrayItem4",
  "outer": "outerDescription2",
  "inner": "innerDescription2",
  "otherProperties": 2
}
    Login or Signup to reply.
  3. 0

    Here’s a straightforward solution using to_entries:

      to_entries[]
  | .key as $outer
  | .value
  | to_entries[]
  | .key as $inner
  | .value
  | .otherProperties as $otherProperties
  | .items[]
  | {item: ., $outer, $inner, $otherProperties}

    Or, equivalently but without variables:

      to_entries[]
  | {outer: .key} +
    ( .value | to_entries[] | {inner: .key} +
    ( .value | {otherProperties} +
    ( .items[] | {item: .} )))

    If the order of keys is important, you could (for example) add
    an expression such as {item, outer, inner, otherProperties} .

    Addendum: To retain all the keys at the "otherProperties" level (apart from .items), you could tweak the solution above:

      to_entries[]
  | {key} +
    ( .value | to_entries[] | {key} +
    ( .value | del(.items) +
    (.items[] | {item: .} )))
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search