Question 269493 in Json Question posted in
Our archive of expertly curated questions and answers provides insights and solutions to common problems related to this popular data interchange format. From parsing and manipulating JSON data to integrating it with various programming languages and web services, our archive has got you covered. Start exploring today and take your JSON skills to the next level

Recursively transform JSON and add data with jq

I’d like to use jq to recursively transform arbitrary JSON files:

Input

{
  "id": 123456,
  "name": "Test",
  "more": [1, 2, 3],
  "instance": {
    "id": 987654,
    "name": "Different"
  }
}

Output

[
  {
    "name" : "id",
    "type": "number",
    "value": 123456
  },
  {
    "name": "name",
    "type": "string",
    "value": "Test"
  },
  {
    "name":"more",
    "type": "array",
    "value": [1, 2, 3]
  },
  {
    "name": "instance",
    "type": "object",
    "value": [
      {
        "name": "id",
        "type": "number",
        "value": 987654
      },
      {
        "name": "name",
        "type": "string",
        "value": "Different"
      }
    ]
  }  
]

Transformation Explanation

Each key-value pair should be transformed to an object containing the keys name, type, value. While name should be the original key. type should be the type of the original value perceived by jq’s type operator. Since Key-value pairs are transformed to complex objects, pairs on the same hierarchy become part of the same array.

Is this possible to achieve via jq?

Tags:

2

Answers


  1. 0

    Your requirement almost fits what to_entries is doing. But you need to carry down the type in a top-down recursion (because with a bottom-up approach, like when using walk, the objects beforehand converted to arrays by to_entries become indistinguishable from proper arrays when determining the type afterwards). So, a recursive function could set up type and value, and in case value is an object, use to_entries and prepend the name to it recursion results:

    def f: {
  type: type,
  value: (objects |= (to_entries | map({name: .key} + (.value | f))))
};
f.value

    [
  {
    "name": "id",
    "type": "number",
    "value": 123456
  },
  {
    "name": "name",
    "type": "string",
    "value": "Test"
  },
  {
    "name": "more",
    "type": "array",
    "value": [
      1,
      2,
      3
    ]
  },
  {
    "name": "instance",
    "type": "object",
    "value": [
      {
        "name": "id",
        "type": "number",
        "value": 987654
      },
      {
        "name": "name",
        "type": "string",
        "value": "Different"
      }
    ]
  }
]

    Demo

    A note regarding your comment that arrays should "stay the same": While this clarified that arrays should not be transformed (e.g. using numeric indices as their items’ "names"), it’s still unclear whether you want array items that are again objects themselves also be transformed or not (while staying inside the untransformed array). For example, change [1, 2, 3] in your example to [1, {"x":2}, 3]. The above assumes that this array should be reproduced as is (Demo). If, however, you want the recursion to be propagated down into array items as well, also update arrays (before objects, to not run into the bottom-up issue), e.g. using arrays |= map(f.value) | objects |= … (Demo), which would produce [1, [{"name": "x", "type": "number", "value": 2}], 3] for the altered example.

    Login or Signup to reply.
  2. 0

    Incase it is useful, here’s a solution that transforms the elements of both arrays and objects to show their types (demo):

    def transform: 
  [to_entries[] | { type : .value.type, name : .key, value : .value.value }];
  
walk({ type : type, value: (if type == "object" then transform end )} )

    For this input:

    {
  "id": 123456,
  "name": "Test",
  "more": [1, 2, { "c" : 1 }],
  "instance": {
    "id": 987654,
    "name": { "a" : 1, "b" : 2 }
  }
}

    It will produce this output:

    {
  "type": "object",
  "value": [
    {
      "type": "number",
      "name": "id",
      "value": 123456
    },
    {
      "type": "string",
      "name": "name",
      "value": "Test"
    },
    {
      "type": "array",
      "name": "more",
      "value": [
        {
          "type": "number",
          "value": 1
        },
        {
          "type": "number",
          "value": 2
        },
        {
          "type": "object",
          "value": [
            {
              "type": "number",
              "name": "c",
              "value": 1
            }
          ]
        }
      ]
    },
    {
      "type": "object",
      "name": "instance",
      "value": [
        {
          "type": "number",
          "name": "id",
          "value": 987654
        },
        {
          "type": "object",
          "name": "name",
          "value": [
            {
              "type": "number",
              "name": "a",
              "value": 1
            },
            {
              "type": "number",
              "name": "b",
              "value": 2
            }
          ]
        }
      ]
    }
  ]
}
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search