Debian Question posted in
The oficial documentation can be found here.

Python3 / subprocess.check_output / Timeout does not work for every command – Debian

I’m running Debian 10.8 with Python 3.7.3 and running a subprocess inside a script which I would like to interrupt after some seconds. This works for cmd_1 but not for cmd_2 in the example below (timeout is never triggered):

import subprocess
import os, sys

# collect proxies and verify
try:
    # cmd_1 = "while true; do sleep 1; done"
    cmd_2 = "proxybroker find --types HTTPS -l 1000 --outfile proxybroker.txt"
    timeout_sec = 3
    subprocess.check_output(cmd_2, shell=True, timeout=timeout_sec)
except Exception as e:
    print(e)

If I run cmd_2 in a bash, it works fine.
I did install the proxybroker package with pip3 install proxybroker.

On another system running Ubuntu 21.04 the timeout works for both commands. Thanks if someone can provide a hint.

Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    As a workaround, timeout as shell command can be used:

    cmd_2 = "timeout 3s proxybroker find --types HTTPS -l 1000 --outfile proxybroker.txt"

  2. 0

    I can’t precisely explain why this happens, but getting rid of the gratuitous shell fixes it.

    tripleee@debian-buster-docker$ cat >subn.py
import subprocess
# import os, sys  # unused import

# collect proxies and verify
try:
  subprocess.check_output(['proxybroker', 'find', '--types', 'HTTPS', '-l', '1000', '--outfile', 'proxies.txt'], timeout=3)
except Exception as e:
  print(e)
^D
tripleee@debian-buster-docker$ python3 subn.py 
Command '['proxybroker', 'find', '--types', 'HTTPS', '-l', '1000', '--outfile', 'proxies.txt']' timed out after 3 seconds

    I notice that your original attempt does print the timeout message as part of the traceback when I interrupt it with a KeyboardInterrupt.

    tripleee@debian-buster-docker$ python3 sub.py
^CTraceback (most recent call last):
  File "/usr/lib/python3.7/subprocess.py", line 474, in run
    stdout, stderr = process.communicate(input, timeout=timeout)
  File "/usr/lib/python3.7/subprocess.py", line 939, in communicate
    stdout, stderr = self._communicate(input, endtime, timeout)
  File "/usr/lib/python3.7/subprocess.py", line 1682, in _communicate
    self._check_timeout(endtime, orig_timeout)
  File "/usr/lib/python3.7/subprocess.py", line 982, in _check_timeout
    raise TimeoutExpired(self.args, orig_timeout)
subprocess.TimeoutExpired: Command 'proxybroker find --types HTTPS -l 1000 --outfile proxybroker.txt' timed out after 3 seconds

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "sub.py", line 9, in <module>
    subprocess.check_output(cmd_2, shell=True, timeout=timeout_sec)
  File "/usr/lib/python3.7/subprocess.py", line 395, in check_output
    **kwargs).stdout
  File "/usr/lib/python3.7/subprocess.py", line 477, in run
    stdout, stderr = process.communicate()
  File "/usr/lib/python3.7/subprocess.py", line 939, in communicate
    stdout, stderr = self._communicate(input, endtime, timeout)
  File "/usr/lib/python3.7/subprocess.py", line 1681, in _communicate
    ready = selector.select(timeout)
  File "/usr/lib/python3.7/selectors.py", line 415, in select
    fd_event_list = self._selector.poll(timeout)
KeyboardInterrupt
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search