Enable hardware acceleration Nvidia Jetsons for image to video encoding

Question

I want to encode jpg/png images into h264/h265 mp4 video (h265 is preferred if possible).

I tried using the commands of this question: How to create a mp4 video file from PNG images using Gstreamer

I got a mp4 video out with this command:

 gst-launch-1.0 -e multifilesrc location="IMG%03d.png" index=1 caps="image/png,framerate=30/1" ! pngdec ! videoconvert ! omxh265enc ! qtmux ! filesink location=image2.mp4

or

 gst-launch-1.0 -e multifilesrc location="IMG%03d.png" index=1 caps="image/png,framerate=30/1" ! pngdec ! videoconvert ! queue ! x264enc ! queue ! mp4mux ! filesink location=image3.mp4

However according to the docs:

Accelerated_GStreamer_User_Guide

We can have hardware acceleration with:

     H.265 Encode (NVIDIA Accelerated Encode)

gst-launch-1.0 nvarguscamerasrc ! \
 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, \
format=(string)NV12, framerate=(fraction)30/1' ! nvv4l2h265enc \
bitrate=8000000 ! h265parse ! qtmux ! filesink \
 location=<filename_h265.mp4> -e

I changed it a little bit for images as input:

 gst-launch-1.0 multifilesrc location="IMG%03d.png" index=1 caps="image/png,framerate=30/1" ! pngdec ! videoconvert ! queue ! nvv4l2h265enc bitrate=8000000 ! h265parse ! qtmux ! filesink  location=output.mp4 -e

However I get the error:

 WARNING: erroneous pipeline: could not link queue0 to nvv4l2h264enc0

According to the docs in nvv4l2h265enc encoder should be available in GStreamer version 1.0

What I'm I doing wrong?

Answer 1

NVIDIA's devtalk forum is the best place for these sorts of questions, but multifilesrc probably puts images in normal CPU memory, not in the GPU NvBuffers that the nvv4l2h265enc element expects. Furthermore, the encoder only seems to work with NV12-formatted YCbCr data while I think the multifilesrc probably outputs in RGB.

The nvvidconv element converts between the "CPU" parts and the "NVIDIA accelerated" parts by moving the data to GPU memory and converting the color space to NV12.

This launch string worked for me:

gst-launch-1.0 \
multifilesrc location="IMG%03d.png" index=1 caps="image/png,framerate=30/1" ! pngdec \
! nvvidconv \
! 'video/x-raw(memory:NVMM), format=(string)NV12 \
! queue \
! nvv4l2h265enc bitrate=8000000 \
! h265parse \
! qtmux \
! filesink location=output.mp4 -e

The caps string after nvvidconv isn't acutally necessary (I also ran successfully without it). nvv4l2h265enc also provides caps and nvvidconv knows how to change what needs to be changed (color space and memory type). I added it for illustration purposes to let you know what is actually going on.

I hope this helps!