How Can I Stream Subprocess Output in Real Time in Python?

Streaming Subprocess Output While Process is Running

In Python, the execute() method is commonly used to execute external commands. However, by default, it waits for the process to complete before returning the combined stdout and stderr output. This can be undesirable for long-running processes.

To enable real-time printing of process output, you can harness the power of iterators and Python's universal_newlines option. Consider this example:

<code class="python">from __future__ import print_function # Only Python 2.x
import subprocess

def execute(cmd):
    popen = subprocess.Popen(cmd, stdout=subprocess.PIPE, universal_newlines=True)
    for stdout_line in iter(popen.stdout.readline, ''):
        yield stdout_line
    popen.stdout.close()
    return_code = popen.wait()
    if return_code:
        raise subprocess.CalledProcessError(return_code, cmd)</code>

This enhanced execute() function launches the process using Popen, ensuring that stdout is piped and newlines are handled automatically (universal_newlines=True). It then uses an iterator (iter(popen.stdout.readline, '')) to traverse stdout line by line.

This approach allows you to stream output from the process as it becomes available within a loop, enabling you to display real-time progress or respond to intermediate outputs accordingly.

The above is the detailed content of How Can I Stream Subprocess Output in Real Time in Python?. For more information, please follow other related articles on the PHP Chinese website!