'Python subprocess.Popen: non-blocking read of partial lines
My application launches a second application as a sub-process.
I use subprocess.POpen to do a non-blocking read of the second application's STDOUT and STDERR.
The code I inherited works fine to retrieve complete lines (those ending with a newline character). But I can't find any way of retrieving an incomplete line (not ending with a newline character).
To use a simple example, streamlink writes a lot of them, in the following formats:
File output.mp4 already exists! Overwrite it? [y/N] [download][output.mp4] Written 9.5 MB (36s @ 132.6 KB/s)
How do I modify the script below to retrieve un-terminated lines from the subprocess STDOUT and STDERR?
(For brevity, I removed the second half of the DoStuff() class, which neatly closes the subprocess when finished.)
#!/usr/bin/env python3
import os
import queue
import subprocess
import threading
class DoStuff():
# Use streamlink (https://streamlink.github.io/) to download a NASA stream
def __init__(self):
url = 'https://www.youtube.com/watch?v=21X5lGlDOfg'
cmd_list = ['streamlink', '--hls-live-restart', '-o', 'output.mp4', url, 'best']
# Start the child process
stdout_reader = PipeReader()
stderr_reader = PipeReader()
child_process = subprocess.Popen(
cmd_list,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
preexec_fn=os.setsid,
startupinfo=None,
)
stdout_reader.attach_fh(child_process.stdout)
stderr_reader.attach_fh(child_process.stderr)
class PipeReader(threading.Thread):
# Non-blocking read of STDOUT and STDERR
def __init__(self):
super(PipeReader, self).__init__()
self.fh = None
def attach_fh(self, fh):
self.fh = fh
for line in iter(self.fh.readline, str('')):
if line == b'':
break
else:
print(line)
# Let's go
DoStuff()
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
