Question 109568 in Json Question posted in
Our archive of expertly curated questions and answers provides insights and solutions to common problems related to this popular data interchange format. From parsing and manipulating JSON data to integrating it with various programming languages and web services, our archive has got you covered. Start exploring today and take your JSON skills to the next level

Json – Parse curl response in bash without jq

I’m trying to parse an array, child of array, in json response returned from a curl request in bash. Unfortunately i can’t use jq because it can’t be installed on production servers.
I’ve try using awk or sed in curl request but result is empty. So i’m using grep with regex but i can’t iterate properly.

#!/bin/bash

PARAM=$1

# Run environment variables
export $(grep -v '^#' ${PWD}/.env | xargs)

# Step 1 : Get JWT Token
TOKEN=$(curl 
  --silent 
  --request POST "${URL_API}"/login 
  --header 'Content-Type: application/json' 
  --header 'Accept: application/json' 
  -d '{"username": "'${API_USERNAME}'", "password": "'${API_PWD}'" }' 
  | sed -n 's|.*"token":"([^"]*)".*|1|p'
)

# Step 2 curl GET request and put it in json file
JSON_RESULT=$(curl 
  --silent 
  --output test.json 
  --header 'Content-Type: application/json' 
  --header 'Accept: application/json' 
  --header 'Authorization: Bearer '${TOKEN}'' 
  --request GET "${URL_API}"/sites/list?value="${PARAM}" 
)

Json response from API :

{
  "data": [
    {
      "id": 212,
      "list": [
        "value1",
        "value2",
        "value3",
        "value4"
      ],

      "createdAt": "2023-02-09T15:47:38+01:00",
      "foo": "bar",
      "foo2": "ba2r",
    }
  ],
  "metas": {
    "code": 200,
    "count": 1,
    "total": 5
  }
}

I want as array type the result of "list" key.
I’m trying like this :

# Step 3 retrieve data in "list" sub-key in json
TEST=$(grep -zoP '"list":s*K[^s]*(?=s*,)' test.json)
echo ${TEST}

Display :

> scripts/test.sh: line 35: warning: command substitution: ignored null byte in input
["value1","value2","value3","value4"]

How can parse this as array for doing something like this :

# step 4 iterate
IFS=',' read -ra ADDR <<< "$TEST"
for i in "${ADDR[@]}"; do
  echo $i
done

?

Tags:

3

Answers


  1. Chosen as BEST ANSWER
    0

    Helped with @Paul Hodges solution i've found a way added with this link

    So:

    # Paul Hodges solution above
LIST=$(sed -nE 'H; ${x; s/[ n]+//g; s/^.*"list":[/[/;s/].*/]/;p;}' test.json)

# Link
borkstring=$(echo $LIST | sed -e 's/[//g' -e 's/]//g' -e 's/, //g')
IFS=',' read -ra arr <<< "$borkstring"
for i in ${arr[@]}; do
    VALUE=${i//"}
    echo "value: "$VALUE
done

    Result :

    value: value1
value: value2
value: value3
value: value4

    And it's ok. maybe my code can be beautify more again :)


  2. 0

    Use while read construction:

    while read -r line; do

    [[ $line =~ list ]] && { add_to__list=true; continue; }
    [[ $add_to__list ]] && {
        [[ $line =~ ], ]] && break
        list+=( "${line//[",]}" )
    }

done < json.file

echo "${list[@]}"

    Result:

    value1 value2 value3 value4
    Login or Signup to reply.
  3. 0

    With sed, using a file as input –

    $: sed -nE 'H; ${x; s/[ n]+//g; s/^.*"list":[/[/;s/].*/]/;p;}' file
["value1","value2","value3","value4"]

    This works even if the file has no embedded spaces or newlines.

    Breakout:

    -nE just stacks -n and -E.
    -n says no output unless explicitly requested.
    -E ues Extended pattern matching, so you don’t have to backslash-quote your square brackets.

    In the actual command list,
    H; stacks each line of input onto the Hold buffer.
    No other commands are executed on any line but the last.

    ${...} means "on the last line, do these commands…"

    x exchanges the input buffer and the Hold buffer, putting all those lines you saved into the workspace.

    s/[ n]+//g will substitute spaces and newlines with nothing globally across the buffer, stacking it all into one long line with any spaces removed.

    s/^.*"list":[/[/ says replace everything from the start(^) through the literal list:[ with just [.

    s/].*/]/ says replace everything from the remaining first [ on with just ].

    p will print the workspace buffer, which at this point should just be the data you asked for.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search