CentOS Question posted in
The official documentation can be found here.

subprocess.Popen() sends awk and grep lines differently than expected – CentOS

On a CentOS 7.2 I have a file called cpuload, which contains the latest CPU load data in the following format:

last 30 sec:
average load: 0
cpu0 total load:     0
cpu1 total load:     0
cpu2 total load:     0
cpu3 total load:     1
cpu4 total load:     0
cpu5 total load:     0
cpu6 total load:     0
cpu7 total load:     0

last sec:
average load: 1
cpu0 total load:     5
cpu1 total load:     1
cpu2 total load:     1
cpu3 total load:     3
cpu4 total load:     2
cpu5 total load:     1
cpu6 total load:     0
cpu7 total load:     0

I want to get the number after the "average load:" of the "last sec" bit.
Two cli commands give me that information when I run them as shell commands on the terminal:
grep 'average load:' cpuload | sed -n 's/.*load: //p' | tail -n1
and
awk 'NR > 2 && /average load:/ {print $3}' cpuload

But when I run them in subprocess.Popen() with Shell=True I only get stderr:

for:

import subprocess

cmd = ["grep", "'average load:'", "cpuload", "|", "sed", "-n", "'s/.*load: //p'", "|", "tail", "-n1"]
test = subprocess.Popen(cmd, stdout = subprocess.PIPE, stderr = subprocess.PIPE, shell=True)
test.stderr.read()

I get:

b"Usage: grep [OPTION]... PATTERN [FILE]...nTry 'grep --help' for more information.n"

and for:

import subprocess
    
cmd = cmd = ["awk", "'NR > 2 && /average load:/ {print $3}'", "cpuload"]
test = subprocess.Popen(cmd, stdout = subprocess.PIPE, stderr = subprocess.PIPE)
test.stderr.read()

I also get:

b"awk: cmd. line:1: 'NR > 2 && /average load:/ {print $3}'nawk: cmd. line:1: ^ invalid char ''' in expressionn"

Even though I avoided using a |

or if shell=True I get:

b"Usage: awk [POSIX or GNU style options] -f progfile [--] file ...nUsage: awk [POSIX or GNU style options] [--] 'program' file ...nPOSIX options:ttGNU long options: (standard)nt-f progfilett--file=progfilent-F fsttt--field-separator=fsnt-v var=valtt--assign=var=valnShort options:ttGNU long options: (extensions)nt-bttt--characters-as-bytesnt-cttt--traditionalnt-Cttt--copyrightnt-d[file]tt--dump-variables[=file]nt-e 'program-text't--source='program-text'nt-E filettt--exec=filent-gttt--gen-potnt-httt--helpnt-L [fatal]tt--lint[=fatal]nt-nttt--non-decimal-datant-Nttt--use-lc-numericnt-Ottt--optimizent-p[file]tt--profile[=file]nt-Pttt--posixnt-rttt--re-intervalnt-Sttt--sandboxnt-tttt--lint-oldnt-Vttt--versionnnTo report bugs, see node `Bugs' in `gawk.info', which isnsection `Reporting Problems and Bugs' in the printed version.nngawk is a pattern scanning and processing language.nBy default it reads standard input and writes standard output.nnExamples:ntgawk '{ sum += $1 }; END { print sum }' filentgawk -F: '{ print $1 }' /etc/passwdn"

What am I doing wrong?

Tags:

3

Answers


  1. Chosen as BEST ANSWER
    0

    The issue was with passing the awk parameter to subprocess with ' around them, as detailed here

    Did not accept Ed Morton's comment as it did not specify what should have been done.


  2. 0

    First case with grep, sed, tail… and pipes.
    You need to use shell = True parameter for Popen method and a single string for the command. We need to put cotes around parameters:

    import subprocess
cmd = "grep 'average load:' cpuload | sed -n 's/.*load: //p' | tail -n1"
output = ""
test = subprocess.Popen(cmd, stdout = subprocess.PIPE, stderr = subprocess.PIPE, shell = True)
while True:
    output += test.stdout.readline().decode("utf-8")
    if test.poll() is not None:
        break
print("output=<%s>" % (output))

    Second case, without pipe:
    You don’t need to use shell = True parameter for Popen method and a single string for the command. We don’t put cotes around parameters:

    import subprocess

cmd = ["/usr/bin/awk", "NR > 2 && /^average load:/ {print $3}", "cpuload"]
output = ""
test = subprocess.Popen(cmd, stdout = subprocess.PIPE, stderr = subprocess.PIPE)
while True:
    output += test.stdout.readline().decode("utf-8")
    if test.poll() is not None:
        break
print("output=<%s>" % (output))
    Login or Signup to reply.
  3. 0

    I have a file called cpuload, which contains the latest CPU load data …I want to get the number after the "average load:" of the "last sec" bit

    why not just to use simple python code in order to get the value you are looking for?

    with open('cpuload') as f:
    lines = [l.strip() for l in f.readlines()]
    got_it = False
    for line in lines:
        if got_it:
            parts = line.split(':')
            result = parts[-1].strip()
            print(result)
            break
        if line == 'last sec:':
            got_it = True

    output

    1
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search