Json Question posted in
Our archive of expertly curated questions and answers provides insights and solutions to common problems related to this popular data interchange format. From parsing and manipulating JSON data to integrating it with various programming languages and web services, our archive has got you covered. Start exploring today and take your JSON skills to the next level

Flatten and filter multi-dimension JSON with JQ

I have the following JSON test.json:

[
  {
    "status": {
      "+@state": "up"
    },
    "address": {
      "+@addr": "10.10.10.1",
      "+@addrtype": "ipv4"
    },
    "ports": {
      "port": [
        {
          "+@protocol": "tcp",
          "+@portid": "80",
          "state": {
            "+@state": "open"
          }
        },
        {
          "+@protocol": "tcp",
          "+@portid": "443",
          "state": {
            "+@state": "closed"
          }
        }
      ]
    }
  },
  {
    "status": {
      "+@state": "down"
    },
    "address": {
      "+@addr": "10.10.10.2",
      "+@addrtype": "ipv4"
    },
    "ports": {
      "port": [
        {
          "+@protocol": "tcp",
          "+@portid": "21",
          "state": {
            "+@state": "closed"
          }
        },
        {
          "+@protocol": "tcp",
          "+@portid": "22",
          "state": {
            "+@state": "closed"
          }
        }
      ]
    }
  },
  {
    "status": {
      "+@state": "up"
    },
    "address": {
      "+@addr": "10.10.10.3",
      "+@addrtype": "ipv4"
    },
    "ports": {
      "port": [
        {
          "+@protocol": "tcp",
          "+@portid": "21",
          "state": {
            "+@state": "closed"
          }
        },
        {
          "+@protocol": "tcp",
          "+@portid": "22",
          "state": {
            "+@state": "closed"
          }
        }
      ]
    }
  },
  {
    "status": {
      "+@state": "up"
    },
    "address": {
      "+@addr": "10.10.10.4",
      "+@addrtype": "ipv4"
    },
    "ports": {
      "port": [
        {
          "+@protocol": "tcp",
          "+@portid": "21",
          "state": {
            "+@state": "open"
          }
        },
        {
          "+@protocol": "tcp",
          "+@portid": "22",
          "state": {
            "+@state": "open"
          }
        }
      ]
    }
  }
]

Running jq --null-input '[ inputs | .[] | select( .status."+@state" == "up" and .ports.port[].state."+@state" == "open" ) | { address: .address."+@addr", port: .ports.port } ]' test.json gets me the following:

[
  {
    "address": "10.10.10.1",
    "port": [
      {
        "+@protocol": "tcp",
        "+@portid": "80",
        "state": {
          "+@state": "open"
        }
      },
      {
        "+@protocol": "tcp",
        "+@portid": "443",
        "state": {
          "+@state": "closed"
        }
      }
    ]
  },
  {
    "address": "10.10.10.4",
    "port": [
      {
        "+@protocol": "tcp",
        "+@portid": "21",
        "state": {
          "+@state": "open"
        }
      },
      {
        "+@protocol": "tcp",
        "+@portid": "22",
        "state": {
          "+@state": "open"
        }
      }
    ]
  },
  {
    "address": "10.10.10.4",
    "port": [
      {
        "+@protocol": "tcp",
        "+@portid": "21",
        "state": {
          "+@state": "open"
        }
      },
      {
        "+@protocol": "tcp",
        "+@portid": "22",
        "state": {
          "+@state": "open"
        }
      }
    ]
  }
]

How do I flatten the port to one object per port, with the address and filter where the port state == "open"? For example, I would like the output to be:

[
  {
    "address": "10.10.10.1",
    "port": "80"
  },
  {
    "address": "10.10.10.4",
    "port": "21"
  },
  {
    "address": "10.10.10.4",
    "port": "22"
  }
]
Tags:

2

Answers


  1. 0

    You’re looking for something like this:

    map(select(.status."+@state" == "up") | {
  address: .address."+@addr",
  port: .ports.port[] | select(.state."+@state" == "open") ."+@portid"
})

    This doesn’t need --null-input.

    Login or Signup to reply.
  2. 0

    Since you know how to filter and rename keys, I’m going to address the first part of your question: How do I flatten the port to one object per port?

    jq 'map(.address+.ports.port[])' test.json

    Output:

    [
  {
    "+@addr": "10.10.10.1",
    "+@addrtype": "ipv4",
    "+@protocol": "tcp",
    "+@portid": "80",
    "state": {
      "+@state": "open"
    }
  },
  {
    "+@addr": "10.10.10.1",
    "+@addrtype": "ipv4",
    "+@protocol": "tcp",
    "+@portid": "443",
    "state": {
      "+@state": "closed"
    }
  },
  {
    "+@addr": "10.10.10.2",
    "+@addrtype": "ipv4",
    "+@protocol": "tcp",
    "+@portid": "21",
    "state": {
      "+@state": "closed"
    }
  },
  {
    "+@addr": "10.10.10.2",
    "+@addrtype": "ipv4",
    "+@protocol": "tcp",
    "+@portid": "22",
    "state": {
      "+@state": "closed"
    }
  },
  {
    "+@addr": "10.10.10.3",
    "+@addrtype": "ipv4",
    "+@protocol": "tcp",
    "+@portid": "21",
    "state": {
      "+@state": "closed"
    }
  },
  {
    "+@addr": "10.10.10.3",
    "+@addrtype": "ipv4",
    "+@protocol": "tcp",
    "+@portid": "22",
    "state": {
      "+@state": "closed"
    }
  },
  {
    "+@addr": "10.10.10.4",
    "+@addrtype": "ipv4",
    "+@protocol": "tcp",
    "+@portid": "21",
    "state": {
      "+@state": "open"
    }
  },
  {
    "+@addr": "10.10.10.4",
    "+@addrtype": "ipv4",
    "+@protocol": "tcp",
    "+@portid": "22",
    "state": {
      "+@state": "open"
    }
  }
]
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search