set a custom timeout for opencv 'VideoCapture.read' function

Question

I am working on a script that reads a rtps stream from a camera. The problem is that sometimes the connection is not perfect and the frames take a while to arrive. From what I have found the read function from cv2.VideoCapture does not have a timeout that we can modify without recompiling, and the default (30 seconds) is way too much for what I need.

I tried two approaches, one using threading and the other using multiprocessing. The former didn't work as expected since I cannot kill the thread fast enough and the script dies. the latter means that I am creating and destroying processes at a rate of 1/fps when everything is working, which I don't think is a good idea.

The following is a minimum working example. When proc = True, it uses multiprocessing, and when proc = False, it uses threading. the delay of the read function can be mimicked using TIMESLEEP > 0

import cv2
import time
import queue
import psutil
import threading
import multiprocessing as mp

TIMESLEEP = 0

class FrameThread(threading.Thread):
    def __init__(self, func, res):
        super().__init__()
        self.daemon = True
        self.res  = res
        self.func = func

    def run(self):
        time.sleep(TIMESLEEP)
        self.res.put(self.func)

def putframe(func, res):
    time.sleep(TIMESLEEP)
    res.put(func)

class Test(object):
    def __init__(self, url, proc = True):
        self.url   = url
        self.black = [1, 2, 3]
        self.fps   = 10
        self.proc  = proc
        self._rq   = mp.Queue() if self.proc else queue.Queue()

    def _timeout_func(self, func, timeout = 10):
        if self.proc:
            _proc = mp.Process(target = putframe, args = (func, self._rq))
            _proc.start()
        else:
            FrameThread(func, self._rq).start()
        try:
            t1  = time.time()
            ret, frame = self._rq.get(block = True, timeout = timeout)
            diff_fps = 1 / self.fps - (time.time() - t1)
            time.sleep(diff_fps if diff_fps > 0 else 0)
            if self.proc:
                _proc.terminate()
            frame = frame if ret else self.black.copy()
        except queue.Empty:
            diff_fps = 1 / self.fps - timeout
            time.sleep(diff_fps if diff_fps > 0 else 0)
            if self.proc:
                _proc.terminate()
            ret, frame = True, self.black.copy()
        return ret, frame


    def run(self):
        cap  = cv2.VideoCapture(self.url)
        while True:
            ret, frame = self._timeout_func(cap.read(), timeout = 0.1)
            if not ret:
                break
            print(self.proc if self.proc else len(psutil.Process().threads()), end='\r')

proc = False
test = Test('./video.mp4', proc = proc)
test.run()

Do you guys have any other idea or approach to do this? or any improvement on the above code?

Thanks!

Answer 1

Not tried this sort of script but I saw a similar kind of question and I would suggest you to use the e VLC python bindings (you can install it with pip install python-vlc) and play the stream:

import vlc
player=vlc.MediaPlayer('rtsp://:8554/output.h264')
player.play()

Then take a snapshot every second or so:

while 1:
    time.sleep(1)
    player.video_take_snapshot(0, '.snapshot.tmp.png', 0, 0)

And then you can use SimpleCV or something for processing (just load the image file '.snapshot.tmp.png' into your processing library).