Play mp4 video using GStreamer on Google Coral Dev Board Mini

Question

I'm trying to play a video file on a Google Coral Dev Board Mini using GStreamer.

I am using the file "drop.avi" from here: https://www.engr.colostate.edu/me/facil/dynamics/avis.htm

I can successfully play this file using the following command:

gst-launch-1.0 playbin uri=file:///home/mendel/coral/google-coral/example-object-tracker/gstreamer/drop.avi

I now want to try playing an mp4 file.

I have converted the avi file using the following command from here:

ffmpeg -i drop.avi -an -vf "scale=960:540,format=yuv420p" -colorspace bt470bg -color_range tv -color_primaries smpte170m -color_trc bt709 -vcodec libx264 -brand mp42 -c:v libx264 -profile:v baseline drop.mp4

and get the following:

$ mediainfo drop.mp4 
General
Complete name                            : drop.mp4
Format                                   : MPEG-4
Format profile                           : Base Media / Version 2
Codec ID                                 : mp42 (isom/iso2/avc1/mp41)
File size                                : 1.82 MiB
Duration                                 : 6 s 67 ms
Overall bit rate                         : 2 511 kb/s
Writing application                      : Lavf58.20.100

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : [email protected]
Format settings                          : 3 Ref Frames
Format settings, CABAC                   : No
Format settings, ReFrames                : 3 frames
Codec ID                                 : avc1
Codec ID/Info                            : Advanced Video Coding
Duration                                 : 6 s 67 ms
Bit rate                                 : 2 509 kb/s
Width                                    : 960 pixels
Height                                   : 540 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 30.000 FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.161
Stream size                              : 1.81 MiB (100%)
Writing library                          : x264 core 155 r2917 0a84d98
Encoding settings                        : cabac=0 / ref=3 / deblock=1:0:0 / analyse=0x1:0x111 / me=hex / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=0 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=6 / lookahead_threads=1 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=0 / weightp=0 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=crf / mbtree=1 / crf=23.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00
Color range                              : Limited
Color primaries                          : BT.601 NTSC
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.470 System B/G
Codec configuration box                  : avcC

But when I try to play this I get an error:

$ gst-launch-1.0 playbin uri=file:///home/mendel/coral/google-coral/example-object-tracker/gstreamer/drop.mp4
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Got context from element 'sink': gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayWayland\)\ gldisplaywayland0";
Got context from element 'playsink': gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayWayland\)\ gldisplaywayland0";
WARNING: from element /GstPlayBin:playbin0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstCapsFilter:capsfilter0: not negotiated
Additional debug info:
gstbasetransform.c(1415): gst_base_transform_reconfigure (): /GstPlayBin:playbin0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstCapsFilter:capsfilter0:
not negotiated
ERROR: from element /GstPlayBin:playbin0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstQTDemux:qtdemux0: Internal data stream error.
Additional debug info:
qtdemux.c(6073): gst_qtdemux_loop (): /GstPlayBin:playbin0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstQTDemux:qtdemux0:
streaming stopped, reason not-negotiated (-4)
ERROR: pipeline doesn't want to preroll.
Setting pipeline to NULL ...
Freeing pipeline ...

What am I doing wrong?

Edit:

The following works:

gst-launch-1.0 filesrc location=drop.mp4 ! decodebin ! autovideoconvert ! autovideosink

But this doesn't

$ gst-launch-1.0 filesrc location=drop.mp4 ! decodebin ! videoconvert ! autovideosink
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Got context from element 'autovideosink0': gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayWayland\)\ gldisplaywayland0";
WARNING: from element /GstPipeline:pipeline0/GstDecodeBin:decodebin0: Delayed linking failed.
Additional debug info:
./grammar.y(510): gst_parse_no_more_pads (): /GstPipeline:pipeline0/GstDecodeBin:decodebin0:
failed delayed linking some pad of GstDecodeBin named decodebin0 to some pad of GstVideoConvert named videoconvert0
ERROR: from element /GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstQTDemux:qtdemux0: Internal data stream error.
Additional debug info:
qtdemux.c(6073): gst_qtdemux_loop (): /GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstQTDemux:qtdemux0:
streaming stopped, reason not-linked (-1)
ERROR: pipeline doesn't want to preroll.
Setting pipeline to NULL ...
Freeing pipeline ...

Still quite confused - am I missing something fundamental?

Answer 1

You write you are using a Google development board. Assuming this is some embedded development board, chances are high that the decoder on that board is implemented in hardware. So most likely your final image buffer after decoding resides in GPU memory.

autovideoconvert probably selects a color converter that operates on these GPU buffers (opengl? vaapi? thats something you will find out). videoconvert on the oither hand operates on buffers from the host main memory. So before you could use that you will need to download the video buffer to the host memory - gldownload in case of OpenGL buffers for example.