Question posted in Json
Our archive of expertly curated questions and answers provides insights and solutions to common problems related to this popular data interchange format. From parsing and manipulating JSON data to integrating it with various programming languages and web services, our archive has got you covered. Start exploring today and take your JSON skills to the next level

dynamic use of jq json for http request

user5820327
December 23, 2022
286 views
0 votes
2 Answers

I have been trying to find a way to make sure I can pass the key and values of this json in this link dynamically into influxDB. I have the batch file that is below :

The json file is below from that link :

{
  "status": "UP",
  "WBAD": {
    "status": "UP",
    "count": "0",
    "minDateTime": "",
    "description": "Workbasket Admin"
  },
  "WBCA": {
    "status": "UP",
    "count": "0",
    "minDateTime": "",
    "description": "Workbasket CreateAppWait"
  },
  "WBDE": {
    "status": "UP",
    "count": "0",
    "minDateTime": "",
    "description": "Workbasket default@"
  },
  "WBEW": {
    "status": "UP",
    "count": "0",
    "minDateTime": "",
    "description": "Workbasket eFormWriteFailure"
  },
  "WBFB": {
    "status": "UP",
    "count": "0",
    "minDateTime": "",
    "description": "Workbasket BackgroundProcessing"
  },
  "WBIC": {
    "status": "UP",
    "count": "0",
    "minDateTime": "",
    "description": "Workbasket IncompleteConnections"
  },
  "WBLB": {
    "status": "UP",
    "count": "17",
    "minDateTime": "23/12/2022 14:50",
    "description": "Workbasket LRBackgroundProcess"
  },
  "AEWB": {
    "status": "UP",
    "count": "0",
    "minDateTime": "",
    "description": "Assignment errors for Assign-WorkBasket"
  },
  "AEWL": {
    "status": "UP",
    "count": "0",
    "minDateTime": "",
    "description": "Assignment errors for Assign-Worklist"
  },
  "FEWB": {
    "status": "UP",
    "count": "0",
    "minDateTime": "",
    "description": "Flow errors for Assign-WorkBasket"
  },
  "FEWL": {
    "status": "UP",
    "count": "0",
    "minDateTime": "",
    "description": "Flow errors for Assign-Worklist"
  },
  "BQBP": {
    "status": "UP",
    "count": "0",
    "minDateTime": "",
    "description": "Broken queue System-Queue-BackgroundProcess"
  },
  "BQDE": {
    "status": "UP",
    "count": "0",
    "minDateTime": "",
    "description": "Broken queue System-Queue-DefaultEntry"
  },
  "CQCA": {
    "status": "UP",
    "count": "0",
    "minDateTime": "",
    "description": "Custom query Create App Requests"
  },
  "JS02": {
    "status": "UP",
    "description": "Job Scheduler CaseDocumentDeletion (Any one associated node:BackgroundProcessing)"
  },
  "JS04": {
    "status": "UP",
    "description": "Job Scheduler PurgeOldSIDExchangeRecords (All associated nodes:BackgroundProcessing)"
  },
  "JS01": {
    "status": "UP",
    "description": "Job Scheduler UpdateReferenceData (Any one associated node:BackgroundProcessing)"

}

The batch file :

 #!/bin/bash  
## GET  STATUS
WBLB_STATUS=`curl -s http://example:8080/Cluster | jq -r '.WBLB.status'`

WBCA_STATUS=`curl -s http://example:8080/Cluster | jq -r '.WBCA.status'`
       
if [ "$WBLB_STATUS" = "UP" ]; then
    echo "app_custom,wblb_status="UP" wlb_status_code=1"

elif [ "$WBLB_STATUS" = "DOWN" ]; then
    echo "app_custom,wblb_status="DOWN" wblb_status_code=0"
    
fi    
exit

The output is

app_custom,wblb_status=UP wlb_status_code=1

I do not want to each key with if but would like to have the output like this below

app_custom,app_wblb_status=UP wlb_status_code=1
app_custom,app_wblc_status=UP wlb_status_code=1
app_custom,app_wbad_status=UP wlb_status_code=1
......

I would want to use this to pass to the influx DB using this conf :

[[inputs.exec]]
  commands = ["/etc/telegraf/telegraf.d/app_test.sh"]
  data_format = "influx"
  timeout = "30s"
  interval = "2m"

Tags: bash influxdb jq json telegraf

Answers

You can use to_entries to split up the items into an array of key-value pairs, then string interpolation to piece together the output strings:

curl -s http://example:8080/Cluster | jq -r '
  to_entries[] | select(.value | type == "object")
  | [.key, .value.status] as [$id, $status]
  | "app_custom,($id | ascii_downcase)_status=($status) wlb_status_code=(["DOWN","UP"] | index($status))"
'

app_custom,wbad_status=UP wlb_status_code=1
app_custom,wbca_status=UP wlb_status_code=1
app_custom,wbde_status=UP wlb_status_code=1
app_custom,wbew_status=UP wlb_status_code=1
app_custom,wbfb_status=UP wlb_status_code=1
app_custom,wbic_status=UP wlb_status_code=1
app_custom,wblb_status=UP wlb_status_code=1
app_custom,aewb_status=UP wlb_status_code=1
app_custom,aewl_status=UP wlb_status_code=1
app_custom,fewb_status=UP wlb_status_code=1
app_custom,fewl_status=UP wlb_status_code=1
app_custom,bqbp_status=UP wlb_status_code=1
app_custom,bqde_status=UP wlb_status_code=1
app_custom,cqca_status=UP wlb_status_code=1
app_custom,js02_status=UP wlb_status_code=1
app_custom,js04_status=UP wlb_status_code=1
app_custom,js01_status=UP wlb_status_code=1

Demo

Use the to_entries function to transform your entries into a list of dictionaries with key and value members, and then you have a data structure that is much easier to filter.

That is, running jq to_entries data.json (where data.json contains your sample data) produces output like:

[
  {
    "key": "status",
    "value": "UP"
  },
  {
    "key": "WBAD",
    "value": {
      "status": "UP",
      "count": "0",
      "minDateTime": "",
      "description": "Workbasket Admin"
    }
  },
  ...

We can extract the keys and values we want from that, and then use the @tsv filter to generate output that’s easier to work with in the shell. The following jq command line:

jq -r 'to_entries[]|select(.key != "status")|[.key, .value.status]|@tsv' data.json

Produces as output:

WBAD    UP
WBCA    UP
WBDE    UP
WBEW    UP
WBFB    UP
WBIC    UP
WBLB    UP
AEWB    UP
AEWL    UP
FEWB    UP
FEWL    UP
BQBP    UP
BQDE    UP
CQCA    UP
JS02    UP
JS04    UP
JS01    UP

We can read values using the read shell function:

#!/bin/bash

jq -r 'to_entries[]|select(.key != "status")|[.key, .value.status]|@tsv' data.json |
while read key status; do
    [[ $status = "UP" ]] && code=1 || code=0
    echo "app_custom,${key,,}_status="$status" wlb_status_code=$code"
done

Which produces:

app_custom,wbad_status="UP" wlb_status_code=1
app_custom,wbca_status="UP" wlb_status_code=1
app_custom,wbde_status="UP" wlb_status_code=1
app_custom,wbew_status="UP" wlb_status_code=1
app_custom,wbfb_status="UP" wlb_status_code=1
app_custom,wbic_status="UP" wlb_status_code=1
app_custom,wblb_status="UP" wlb_status_code=1
app_custom,aewb_status="UP" wlb_status_code=1
app_custom,aewl_status="UP" wlb_status_code=1
app_custom,fewb_status="UP" wlb_status_code=1
app_custom,fewl_status="UP" wlb_status_code=1
app_custom,bqbp_status="UP" wlb_status_code=1
app_custom,bqde_status="UP" wlb_status_code=1
app_custom,cqca_status="UP" wlb_status_code=1
app_custom,js02_status="UP" wlb_status_code=1
app_custom,js04_status="UP" wlb_status_code=1
app_custom,js01_status="UP" wlb_status_code=1

Please signup or login to give your own answer.

Click here to cancel reply.