Table of Contents
Stop reading process output in Python without hang?
Spooled Temporary File (Recommended)
Thread-based Output Reading
signal.alarm() (Unix-only)
threading.Timer
No Threads, No Signals
Home Backend Development Python Tutorial How to Avoid Python Programs from Hanging When Reading Process Output?

How to Avoid Python Programs from Hanging When Reading Process Output?

Nov 02, 2024 pm 01:54 PM

How to Avoid Python Programs from Hanging When Reading Process Output?

Stop reading process output in Python without hang?

Problem:

A Python program needs to interact with an external process (e.g., "top") that continuously produces output. However, simply reading the output directly can cause the program to hang indefinitely.

Solution:

To prevent hanging, it's essential to employ non-blocking or asynchronous mechanisms when reading process output. Here are a few possible approaches:

This method utilizes a dedicated file object to store the process output.

<pre>#!/usr/bin/env python
import subprocess
import tempfile
import time

def main():

# Open a temporary file (automatically deleted on closure)
f = tempfile.TemporaryFile()

# Start the process and redirect stdout to the file
p = subprocess.Popen(["top"], stdout=f)

# Wait for a specified duration
time.sleep(2)

# Kill the process
p.terminate()
p.wait()

# Rewind and read the captured output from the file
f.seek(0)
output = f.read()

# Print the output
print(output)
f.close()
Copy after login

if name == "__main__":

main()
Copy after login
Copy after login
Copy after login
Copy after login
Copy after login

</pre>

Thread-based Output Reading

This approach employs a separate thread to continuously read the process output while the main thread proceeds with other tasks.

<pre>import collections
import subprocess
import threading
import time

def read_output(process, append):

for line in iter(process.stdout.readline, ""):
    append(line)
Copy after login

def main():

# Start the process and redirect stdout
process = subprocess.Popen(["top"], stdout=subprocess.PIPE, close_fds=True)

# Create a thread for output reading
q = collections.deque(maxlen=200)
t = threading.Thread(target=read_output, args=(process, q.append))
t.daemon = True
t.start()

# Wait for the specified duration
time.sleep(2)

# Print the saved output
print(''.join(q))
Copy after login

if name == "__main__":

main()
Copy after login
Copy after login
Copy after login
Copy after login
Copy after login

</pre>

signal.alarm() (Unix-only)

This method uses Unix signals to terminate the process after a specified timeout, regardless of whether all output has been read.

<pre>import collections
import signal
import subprocess

class Alarm(Exception):

pass
Copy after login

def alarm_handler(signum, frame):

raise Alarm
Copy after login

def main():

# Start the process and redirect stdout
process = subprocess.Popen(["top"], stdout=subprocess.PIPE, close_fds=True)

# Set signal handler
signal.signal(signal.SIGALRM, alarm_handler)
signal.alarm(2)

try:
    # Read and save a specified number of lines
    q = collections.deque(maxlen=200)
    for line in iter(process.stdout.readline, ""):
        q.append(line)
    signal.alarm(0)  # Cancel alarm
except Alarm:
    process.terminate()
finally:
    # Print the saved output
    print(''.join(q))
Copy after login

if name == "__main__":

main()
Copy after login
Copy after login
Copy after login
Copy after login
Copy after login

</pre>

threading.Timer

This approach employs a timer to terminate the process after a specified timeout. It works on both Unix and Windows systems.

<pre>import collections
import subprocess
import threading

def main():

# Start the process and redirect stdout
process = subprocess.Popen(["top"], stdout=subprocess.PIPE, close_fds=True)

# Create a timer for process termination
timer = threading.Timer(2, process.terminate)
timer.start()

# Read and save a specified number of lines
q = collections.deque(maxlen=200)
for line in iter(process.stdout.readline, ""):
    q.append(line)
timer.cancel()

# Print the saved output
print(''.join(q))
Copy after login

if name == "__main__":

main()
Copy after login
Copy after login
Copy after login
Copy after login
Copy after login

</pre>

No Threads, No Signals

This method uses a simple time-based loop to check for process output and kill it if it exceeds a specified timeout.

<pre>import collections
import subprocess
import sys
import time

def main():

args = sys.argv[1:]
if not args:
    args = ['top']

# Start the process and redirect stdout
process = subprocess.Popen(args, stdout=subprocess.PIPE, close_fds=True)

# Save a specified number of lines
q = collections.deque(maxlen=200)

# Set a timeout duration
timeout = 2

now = start = time.time()
while (now - start) &lt; timeout:
    line = process.stdout.readline()
    if not line:
        break
    q.append(line)
    now = time.time()
else:  # On timeout
    process.terminate()

# Print the saved output
print(''.join(q))
Copy after login

if name == "__main__":

main()
Copy after login
Copy after login
Copy after login
Copy after login
Copy after login

</pre>

Note: The number of lines stored can be adjusted as needed by setting the 'maxlen' parameter of the deque data structure.

The above is the detailed content of How to Avoid Python Programs from Hanging When Reading Process Output?. For more information, please follow other related articles on the PHP Chinese website!

Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn

Hot Article Tags

Notepad++7.3.1

Notepad++7.3.1

Easy-to-use and free code editor

SublimeText3 Chinese version

SublimeText3 Chinese version

Chinese version, very easy to use

Zend Studio 13.0.1

Zend Studio 13.0.1

Powerful PHP integrated development environment

Dreamweaver CS6

Dreamweaver CS6

Visual web development tools

SublimeText3 Mac version

SublimeText3 Mac version

God-level code editing software (SublimeText3)

How Do I Use Beautiful Soup to Parse HTML? How Do I Use Beautiful Soup to Parse HTML? Mar 10, 2025 pm 06:54 PM

How Do I Use Beautiful Soup to Parse HTML?

How to Use Python to Find the Zipf Distribution of a Text File How to Use Python to Find the Zipf Distribution of a Text File Mar 05, 2025 am 09:58 AM

How to Use Python to Find the Zipf Distribution of a Text File

Image Filtering in Python Image Filtering in Python Mar 03, 2025 am 09:44 AM

Image Filtering in Python

How to Perform Deep Learning with TensorFlow or PyTorch? How to Perform Deep Learning with TensorFlow or PyTorch? Mar 10, 2025 pm 06:52 PM

How to Perform Deep Learning with TensorFlow or PyTorch?

Introduction to Parallel and Concurrent Programming in Python Introduction to Parallel and Concurrent Programming in Python Mar 03, 2025 am 10:32 AM

Introduction to Parallel and Concurrent Programming in Python

Serialization and Deserialization of Python Objects: Part 1 Serialization and Deserialization of Python Objects: Part 1 Mar 08, 2025 am 09:39 AM

Serialization and Deserialization of Python Objects: Part 1

How to Implement Your Own Data Structure in Python How to Implement Your Own Data Structure in Python Mar 03, 2025 am 09:28 AM

How to Implement Your Own Data Structure in Python

Mathematical Modules in Python: Statistics Mathematical Modules in Python: Statistics Mar 09, 2025 am 11:40 AM

Mathematical Modules in Python: Statistics

See all articles