Question 109192 in Json Question posted in
Our archive of expertly curated questions and answers provides insights and solutions to common problems related to this popular data interchange format. From parsing and manipulating JSON data to integrating it with various programming languages and web services, our archive has got you covered. Start exploring today and take your JSON skills to the next level

Conditional parsing of Nested Jsons with jq

Need help parsing the below nested Json. The end goal is to get the quota limits from GCP that have non-default values. Below is the sample data for one of the services (compute APIs for a given CPU type)

I am struggling to be able to compare
I came up with the below but it doesn’t work and returns an error
.[]|select (.quota[].defaultLimit|tonumber != .quota[].effectiveLimit|tonumber)

[
  {
    "metric": "compute.googleapis.com/a2_cpus",
    "quota": [
      {
        "defaultLimit": "-1",
        "effectiveLimit": "-1"
      }
    ]
  },
  {
    "metric": "compute.googleapis.com/a2_cpus",
    "quota": [
      {
        "defaultLimit": "12",
        "effectiveLimit": "12"
      },
      {
        "defaultLimit": "12",
        "dimensions": {
          "region": "asia-east1"
        },
        "effectiveLimit": "13",
        "producerOverride": {
          "dimensions": {
            "region": "asia-east1"
          }
        }
      }
    ]
  }
]

the expected output would be something like this

[
  {
    "metric": "compute.googleapis.com/a2_cpus",
    "quota": [    
      {
        "defaultLimit": "12",
        "dimensions": {
          "region": "asia-east1"
        },
        "effectiveLimit": "13"
      }
    ]
  }
]

Any help is appreciated.

Tags:

3

Answers


  1. 0

    Based on your basic description of the problem, it looks like this would suffice:

    walk(if type == "object" and has("defaultLimit") 
     then select(.defaultLimit != .effectiveLimit) else . end)

    However, based on your example, you might want to extend the pipeline by adding:

    | walk(if type == "object" then select(.quota != []) else . end)
    Login or Signup to reply.
  2. 0

    You could first select all quotas with a non-default limit and then drop any metric without remaining quotas/keep only metrics with at least one quota:

    map(.quota |= map(select(.defaultLimit != .effectiveLimit)) | select(.quota | length > 0))

    Output:

    [
  {
    "metric": "compute.googleapis.com/a2_cpus",
    "quota": [
      {
        "defaultLimit": "12",
        "dimensions": {
          "region": "asia-east1"
        },
        "effectiveLimit": "13",
        "producerOverride": {
          "dimensions": {
            "region": "asia-east1"
          }
        }
      }
    ]
  }
]

    To keep only the limits and dimensions (dropping producerOverride and any other additional properties):

    map(.quota |= map(select(.defaultLimit != .effectiveLimit) | {defaultLimit, effectiveLimit, dimensions}) | select(.quota | length > 0))

    Output:

    [
  {
    "metric": "compute.googleapis.com/a2_cpus",
    "quota": [
      {
        "defaultLimit": "12",
        "effectiveLimit": "13",
        "dimensions": {
          "region": "asia-east1"
        }
      }
    ]
  }
]
    Login or Signup to reply.
  3. 0

    A possible solution:

    map(
  .quota |=
    map(
      select(.defaultLimit != .effectiveLimit)
      | del(.producerOverride)
    )
  | select(.quota | length > 0)
)

    How it works

    • map(...) – because the input is an array, map(...) applies the expression passed as argument to each item of the array and yields a new array that contains the results of the expression applied to each argument; all subsequent steps described below operate on one object (each item of the array).
    • .quota |= map(...) – the value of property .quota (which is an array) is passed to map() (which yields a new array) and the resulting array is stored back in .quota; this is the same thing as .quota = (.quota | map(...)) but it’s shorter and easier to read.
    • map(select(...) | del(...)) – for each item of the .quota array run select(.defaultLimit != .effectiveLimit) then remove (del) the property .producerOverride from the returned object (if it exists). The result is an array that contains only the quota entries that non-default values.
    • select(.defaultLimit != .effectiveLimit) – if the condition is met then this yields the input object, otherwise it does not yield anything. This is how the quota items that have non-default values are filtered.
    • del(.producerOverride) – this filter is here because the property .producerOverride is not displayed in your expected output. If the input object may contain other properties, not listed here, and you want to keep only a known set of properties in the output then use this filter instead: 
      | { defaultLimit, dimensions, effectiveLimit }

      But be aware that this one produces objects that always have these three properties, even when not all of them are in the input (dimensions is not present in all quota items in the input). In this case, the value of those properties is null.

    • | select(.quota | length > 0) – this filter gets at input the objects of the input array after their .quota property is processed as described above. It keeps from its input only the objects that have non-empty .quota property. The previous filters produce from the first input item an item that looks like this: 
      {
   "metric": "compute.googleapis.com/a2_cpus",
   "quota": []
}

      This filter removes such items.

    jq documentation

    Read the jq tutorial and the jq manual for more information.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search