Question 110292 in Json Question posted in
Our archive of expertly curated questions and answers provides insights and solutions to common problems related to this popular data interchange format. From parsing and manipulating JSON data to integrating it with various programming languages and web services, our archive has got you covered. Start exploring today and take your JSON skills to the next level

Can you separate distinct JSON attributes into two files using jq?

I am following this tutorial from Vault about creating your own certificate authority. I’d like to separate the response (change the output to API call using cURL to see the response) into two distinct files, one file possessing the certificate and issuing_ca attributes, the other file containing the private_key. The tutorial is using jq to parse JSON objects, but my unfamiliarity with jq isn’t helpful here, and most searches are returning info on how to merge JSON using jq.

I’ve tried running something like

vault write -format=json pki_int/issue/example-dot-com                                                             
common_name="test.example.com"     
ttl="24h"                  
format=pem 
jq -r '.data.certificate, .data.issuing_ca > test.cert.pem 
jq -r '.data.private_key' > test.key.pem

or

vault write -format=json pki_int/issue/example-dot-com                                                             
common_name="test.example.com"     
ttl="24h"                  
format=pem 
| jq -r '.data.certificate, .data.issuing_ca > test.cert.pem 
| jq -r '.data.private_key' > test.key.pem

but no dice.

Tags:

3

Answers


  1. 0

    It is not an issue with jq invocation, but the way the output files get written. Per your usage indicated, after writing the file test.cert.pem, the contents over the read end of the pipe (JSON output) is no longer available to extract the private_key contents.

    To duplicate the contents over at the write end of pipe, use tee along with process substitution. The following should work on bash/zsh or ksh93 and not on POSIX bourne shell sh

    vault write -format=json pki_int/issue/example-dot-com                                                             
common_name="test.example.com"     
ttl="24h"                  
format=pem 
| tee >( jq -r '.data.certificate, .data.issuing_ca' > test.cert.pem) 
>(jq -r '.data.private_key' > test.key.pem) 
>/dev/null

    See this in action

    jq -n '{data:{certificate: "foo", issuing_ca: "bar", private_key: "zoo"}}' 
| tee >( jq -r '.data.certificate, .data.issuing_ca' > test.cert.pem) 
>(jq -r '.data.private_key' > test.key.pem) 
>/dev/null

    and now observe the contents of both the files.

    Login or Signup to reply.
  2. 0

    You could abuse jq‘s ability to write to standard error (version 1.6 or later) separately from standard output.

    vault write -format=json pki_int/issue/example-dot-com                                                             
common_name="test.example.com"     
ttl="24h"                  
format=pem 
| jq -r '.data as $f | ($f.private_key | stderr) | ($f.certificate, $f.issuing_ca)' > test.cert.pem 2> test.key.pem
    Login or Signup to reply.
  3. 0

    There’s a general technique for this type of problem that is worth mentioning
    because it has minimal prerequisites (just jq and awk), and because
    it scales well with the number of files. Furthermore it is quite efficient in that only one invocation each of jq and awk is needed. The idea is to setup a pipeline of the form: jq … | awk …

    There are many variants
    of the technique but in the present case, the following would suffice:

    jq -rc '
 .data
 | "test.cert.pem",
      "t(.certificate)",
      "t(.issuing_ca)", 
   "test.key.pem",
      "t(.private_key)"
' | awk -F\t 'NF == 1 {fn=$1; next} {print $2 > fn}'

    Notice that this works even if the items of interest are strings with embedded tabs.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search