How to Perform Non-Blocking Reads from a Subprocess PIPE?

Reading from a Subprocess PIPE in a Non-Blocking Manner

When working with subprocesses, it can be crucial to perform non-blocking reads to avoid stalling the parent process while waiting for output from the child process. The issue arises when using subprocess.Popen, where readline() blocks if there's no available data.

Traditional Blocking Approach

Typically, developers use the following code to read from the subprocess's standard output:

p = subprocess.Popen('myprogram.exe', stdout = subprocess.PIPE)
output_str = p.stdout.readline()

However, this approach blocks execution if no data is available.

Non-Blocking Solution

A reliable cross-platform solution involves utilizing Queue.get_nowait(), as demonstrated in the code below:

import sys
from subprocess import PIPE, Popen
from threading  import Thread

try:
    from queue import Queue, Empty
except ImportError:
    from Queue import Queue, Empty  # python 2.x

ON_POSIX = 'posix' in sys.builtin_module_names

def enqueue_output(out, queue):
    for line in iter(out.readline, b''):
        queue.put(line)
    out.close()

p = Popen(['myprogram.exe'], stdout=PIPE, bufsize=1, close_fds=ON_POSIX)
q = Queue()
t = Thread(target=enqueue_output, args=(p.stdout, q))
t.daemon = True # thread dies with the program
t.start()

# ... do other things here

# read line without blocking
try:  line = q.get_nowait() # or q.get(timeout=.1)
except Empty:
    print('no output yet')
else: # got line
    # ... do something with line

This approach uses a separate thread to enqueue output lines into a queue. The main thread can then check if the queue contains data without blocking, using get_nowait(). This portable solution allows for non-blocking reads on subprocess outputs across various operating systems.

The above is the detailed content of How to Perform Non-Blocking Reads from a Subprocess PIPE?. For more information, please follow other related articles on the PHP Chinese website!