Question 239721 in Json Question posted in
Our archive of expertly curated questions and answers provides insights and solutions to common problems related to this popular data interchange format. From parsing and manipulating JSON data to integrating it with various programming languages and web services, our archive has got you covered. Start exploring today and take your JSON skills to the next level

Json – sort_by function when using AWS CLI for aws ec2 describe-security-groups

I am trying to compare 2 security groups in AWS that are in 2 different accounts, using aws ec2 describe-security-groups. Even if the security groups are exactly the same, it’s impossible for me to detect that because the json response is sorted differently in the 2 accounts.

SG_1=$(aws ec2 describe-security-groups 
--filters Name="tag:Name",Values="MainSG" 
--query 'SecurityGroups[].{
IpPermissions:IpPermissions[].{
                IpProtocol: IpProtocol,
                IpRanges: IpRanges[].{CidrIp:CidrIp},
                FromPort: FromPort,
                ToPort: ToPort
},
IpPermissionsEgress:IpPermissionsEgress[].{
                IpProtocol: IpProtocol,
                IpRanges: IpRanges[].{CidrIp:CidrIp},
                FromPort: FromPort,
                ToPort: ToPort
}
}' 
--output json)

SG_2=$(aws ec2 describe-security-groups 
--filters Name="tag:Name",Values="MainSG" 
--query 'SecurityGroups[].{
IpPermissions:IpPermissions[].{
                IpProtocol: IpProtocol,
                IpRanges: IpRanges[].{CidrIp:CidrIp},
                FromPort: FromPort,
                ToPort: ToPort
},
IpPermissionsEgress:IpPermissionsEgress[].{
                IpProtocol: IpProtocol,
                IpRanges: IpRanges[].{CidrIp:CidrIp},
                FromPort: FromPort,
                ToPort: ToPort
}
}' 
--output json)

if "$SG_1" = "$SG_2" ]; then
    echo "SAME"
else
    echo "DIFFERENT"
fi

The output for SG_1 is :

[
  {
    "IpPermissions": [
      {
        "IpProtocol": "-1",
        "IpRanges": [
          {
            "CidrIp": "10.192.55.56/32"
          },
          {
            "CidrIp": "10.111.11.0/26"
          }
        ],
        "FromPort": null,
        "ToPort": null
      },

The output for SG_2 is :

[
  {
    "IpPermissions": [
      {
        "IpProtocol": "-1",
        "IpRanges": [
          {
            "CidrIp": "10.111.11.0/26"
          },
          {
            "CidrIp": "10.192.55.56/32"
          }
        ],
        "FromPort": null,
        "ToPort": null
      },

I have tried to use jq but the response is still sorted differently.
Here is the jq command I have tried to use :

jq '.[0].IpPermissions |= sort_by(.IpRanges[].CidrIp) | .[0].IpPermissionsEgress |= sort_by(.IpRanges[].CidrIp)')

I have also tried something simple :

aws ec2 describe-security-groups --query 'sort_by(SecurityGroups, &CidrIp)[].{CidrIp: CidrIp}'

In function sort_by(), invalid type for value:
expected one of: [‘string’, ‘number’], received: "null"

Tags:

2

Answers


  1. 0

    If you’re just trying to sort items in the IpRanges fields, you could do this:

    .[].IpPermissions[].IpRanges |= sort_by(.CidrIp)

    You should be able to then compare for deep equality. You’ll probably want to do this for all arrays that may be ordered differently.

    Login or Signup to reply.
  2. 0

    Try this :

    #!/bin/bash

security-groups(){
aws ec2 describe-security-groups 
    --filters Name="tag:Name",Values="$1" 
    --query 'SecurityGroups[].{
    IpPermissions:IpPermissions[].{
                    IpProtocol: IpProtocol,
                    IpRanges: IpRanges[].{CidrIp:CidrIp},
                    FromPort: FromPort,
                    ToPort: ToPort
    },
    IpPermissionsEgress:IpPermissionsEgress[].{
                    IpProtocol: IpProtocol,
                    IpRanges: IpRanges[].{CidrIp:CidrIp},
                    FromPort: FromPort,
                    ToPort: ToPort
    }
    }' 
    --output json 
| jq '.[][][].IpRanges |= sort_by(.CidrIp)'
}

sg=MainSG

if diff -q <(security-groups $sg) <(security-groups $sg); then
    echo "SAME"
else
    echo "DIFFERENT"
fi

    You need to adapt shell function security-groups in order to get raw data from two different AWS accounts.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search