How to properly receive video via gstreamer in Unity from c++ sender

Question

I am trying to transfer a stream using Opencv using VideoWriter (c++ side using WSL) to get it on another pc on the network using VideoCapture (Unity side).

Inspired by this example: https://opencv94.rssing.com/chan-61447116/article1201-live.html

I want to send a video from OpenCV from C++ to Unity.

So I have this code on the c++ side:

cv::VideoWriter writer;
    writer.open("appsrc ! videoconvert ! x264enc tune=zerolatency bitrate=500 speed-preset=superfast ! rtph264pay ! udpsink host=127.0.0.1 port=5000",
                        cv::CAP_GSTREAMER, 
                        0, 
                        5,
                        cv::Size (640, 480),
                        true);

When I run the c++ program it works and I'm able to use the writer.

on the unity side I have this:

VideoCapture capture = new VideoCapture();
bool opened = capture.open("udpsrc port=5000 ! rtph264pay ! decodebin ! videoconvert ! appsink sync=false", Videoio.CAP_GSTREAMER);

But I always get false on opened so I cant' connect to the c++.

I think the pipelines can be wrong in this case.

Is it really possible to receive a video from gstreamer on Unity?

Note: the c++ side was made using WSL ubuntu 18.06

Answer 1

Main cause may be that your sender is streaming to port 5000 while your receiver tries to read from port 5200. Assuming unity is the same host as writer, you would try:

Sender:

writer.open("appsrc ! videoconvert ! x264enc tune=zerolatency bitrate=500 speed-preset=superfast insert-vui=1 ! h264parse ! rtph264pay config-interval=1 ! udpsink host=127.0.0.1 port=5000 auto-multicast=0",
            cv::CAP_GSTREAMER, 
            0, 
            5.f,
            cv::Size (640, 480),
            true);

Unity receiver:

[EDIT: user @felipe finally reported that OpenCVForUnity is not supported on Unity today 25/mar/2022]

bool opened = capture.open("udpsrc port=5000 ! application/x-rtp,encoding-name=H264 ! rtph264pay ! decodebin ! videoconvert ! appsink sync=false", Videoio.CAP_GSTREAMER);