skip to Main Content

I have a python script that watches a socket and when data is received it writes JSON to stdout.

This sample script below illustrates the behaviour, outputting new JSON every 3 seconds.

import json, time

def new_json(n):
    frame = {
        "Iteration": n,
        "Field0": 0,
        "Field1": 1,
        "Field2": 2
    }
    print(json.dumps(frame))

new_json(1)
time.sleep(3)
new_json(2)
time.sleep(3)
new_json(3)

The output of this is:

{"Iteration": 1, "Field0": 0, "Field1": 1, "Field2": 2}
{"Iteration": 2, "Field0": 0, "Field1": 1, "Field2": 2}
{"Iteration": 3, "Field0": 0, "Field1": 1, "Field2": 2}

If i pipe this into jq using python3 jqtest.py | jq . it only prints the JSON when the script has completely finished.

How can i invoke jq for every new JSON object received during execution, not just at the end?

2

Answers


  1. You need to unbuffer python’s stdout, e.g.:

    python3 -u jqtest.py | jq '.'
    
    Login or Signup to reply.
  2. You can also check this out, by stdbuf -o0, the output buffering is disabled, allowing jq to receive and process the JSON objects immediately as they are generated by the script.

    stdbuf -o0 python3 jqtest.py | jq .
    
    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search