Question 215845 in Json Question posted in
Our archive of expertly curated questions and answers provides insights and solutions to common problems related to this popular data interchange format. From parsing and manipulating JSON data to integrating it with various programming languages and web services, our archive has got you covered. Start exploring today and take your JSON skills to the next level

Json – JQ – group duplicates not remove

I am trying to group the duplicates from the following json by id, but remove duplicates . Here is the json:

{
  "intId": "97210f2c23f45b55569f32309105f395c94",
  "extId": "39e9588ee62d1cccc711d75ff86c5570",
  "name": "generic name",
  "users": {
    "186-1": {
      "userKey": "186-1",
      "extId": "186",
      "intIds": {
        "w_eyvyfghz844o": {
          "intId": "w_eyvyfghz844o",
          "registeredOn": 1690289980777,
          "leftOn": 0,
          "userLeftFlag": false
        }
      },
      "name": "generic name 1",
      "isModerator": true,
      "isDialIn": false,
      "currentIntId": "w_eyvyfghz844o",
      "answers": {},
      "talk": {
        "totalTime": 5696194,
        "lastTalkStartedOn": 1690297856252
      },
      "emojis": [],
      "webcams": [
        {
          "startedOn": 1690289988531,
          "stoppedOn": 0
        }
      ],
      "totalOfMessages": 18
    },
    "95-1": {
      "userKey": "95-1",
      "extId": "95",
      "intIds": {
        "w_kegd1ecwju02": {
          "intId": "w_kegd1ecwju02",
          "registeredOn": 1690295357122,
          "leftOn": 1690295379924,
          "userLeftFlag": true
        },
        "w_nc1egwb2axbr": {
          "intId": "w_nc1egwb2axbr",
          "registeredOn": 1690296171797,
          "leftOn": 0,
          "userLeftFlag": false
        }
      },
      "name": "Generic name 2",
      "isModerator": true,
      "isDialIn": false,
      "currentIntId": "w_nc1egwb2axbr",
      "answers": {},
      "talk": {
        "totalTime": 0,
        "lastTalkStartedOn": 0
      },
      "emojis": [],
      "webcams": [],
      "totalOfMessages": 0
    },
    "95-2": {
      "userKey": "95-2",
      "extId": "95",
      "intIds": {
        "w_et8i4fc0fumu": {
          "intId": "w_et8i4fc0fumu",
          "registeredOn": 1690293905345,
          "leftOn": 1690294109923,
          "userLeftFlag": true
        }
      },
      "name": "Generic name 2",
      "isModerator": true,
      "isDialIn": false,
      "answers": {},
      "talk": {
        "totalTime": 0,
        "lastTalkStartedOn": 0
      },
      "emojis": [],
      "webcams": [],
      "totalOfMessages": 0
    }
  },
  "polls": {},
  "screenshares": [
    {
      "startedOn": 1690290408449,
      "stoppedOn": 1690290573091
    },
    {
      "startedOn": 1690294824305,
      "stoppedOn": 1690294957331
    },
    {
      "startedOn": 1690294976792,
      "stoppedOn": 1690295106468
    },
    {
      "startedOn": 1690295120270,
      "stoppedOn": 1690295742503
    },
    {
      "startedOn": 1690295747135,
      "stoppedOn": 1690297603487
    },
    {
      "startedOn": 1690297609337,
      "stoppedOn": 0
    }
  ],
  "presentationSlides": [
    {
      "presentationId": "8b8e93774c2cebfcde9582e0f9c169",
      "pageNum": 1,
      "setOn": 1690288860858
    }
  ],
  "createdOn": 1690288854908,
  "endedOn": 0
}

Here is what i have tried with:

jq -r '.users | [.[] | { "extId":(.extId),"name":.name, "conections":[( .intIds[] | {"registeredOn":.registeredOn,"leftOn":.leftOn,"userLeftFlag":.userLeftFlag} )] } ] | INDEX(.extId)'

{
  "186": {
    "extId": "186",
    "name": "generic name 1",
    "conections": [
      {
        "registeredOn": 1690289980777,
        "leftOn": 0,
        "userLeftFlag": false
      }
    ]
  },
  "95": {
    "extId": "95",
    "name": "Generic name 2",
    "conections": [
      {
        "registeredOn": 1690293905345,
        "leftOn": 1690294109923,
        "userLeftFlag": true
      }
    ]
  }
}

But using INDEX(.extId) removes the duplicates, so I understand that I need to group them before

[
  {
    "extId": "186",
    "name": "generic name 1",
    "conections": [
      {
        "registeredOn": 1690289980777,
        "leftOn": 0,
        "userLeftFlag": false
      }
    ]
  },
  {
    "extId": "95",
    "name": "Generic name 2",
    "conections": [
      {
        "registeredOn": 1690295357122,
        "leftOn": 1690295379924,
        "userLeftFlag": true
      },
      {
        "registeredOn": 1690296171797,
        "leftOn": 0,
        "userLeftFlag": false
      }
    ]
  },
  {
    "extId": "95",
    "name": "Generic name 2",
    "conections": [
      {
        "registeredOn": 1690293905345,
        "leftOn": 1690294109923,
        "userLeftFlag": true
      }
    ]
  }
]

I don’t require objects to have a key, but I would like to keep it.

Any help would be appreciated.

Tags:

2

Answers


  1. 0

    There are many ways of grouping those users.

    Using the following filter, you can create an output where you change .users to an array holding the grouped users:

    .users |= (to_entries | [ group_by(.value.extId)[] | from_entries ])

    "users": [
    {
      "186-1": { ... }
    },
    {
      "95-1": { ... },
      "95-2": { ... }
    }
]

    Online Demo.

    Login or Signup to reply.
  2. 0

    If you are trying to group by .extId while collecting the objects under .intIDs[], then group_by has to go between the outer and the inner mappings:

    .users
| map({extId, name, connections: .intIds | map({registeredOn, leftOn, userLeftFlag})})
| group_by(.extId)
| map(first + {connections: map(.connections[])})

    Demo

    Alternatively, you could iterate using reduce, group manually on the go by collecting into an object, and restore the array using map(.):

    reduce .users[] as $user ({}; .[$user.extId] |= (($user | {extId, name}) + (
  .connections += [$user.intIds[] | {registeredOn, leftOn, userLeftFlag}]
))) | map(.)

    Demo

    Both approaches output

    [
  {
    "extId": "186",
    "name": "generic name 1",
    "connections": [
      {
        "registeredOn": 1690289980777,
        "leftOn": 0,
        "userLeftFlag": false
      }
    ]
  },
  {
    "extId": "95",
    "name": "Generic name 2",
    "connections": [
      {
        "registeredOn": 1690295357122,
        "leftOn": 1690295379924,
        "userLeftFlag": true
      },
      {
        "registeredOn": 1690296171797,
        "leftOn": 0,
        "userLeftFlag": false
      },
      {
        "registeredOn": 1690293905345,
        "leftOn": 1690294109923,
        "userLeftFlag": true
      }
    ]
  }
]
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search