Create a simple OpenCV pipeline with GStreamer

Question

I have a pipeline to stream images over UDP

fps = 20
width = 500
height = 500

out_send = cv2.VideoWriter(
        "appsrc ! videoconvert ! "
        "video/x-raw,format=I420 ! "
        "jpegenc ! rtpjpegpay !"
        "udpsink host=127.0.0.1 port=5000",
        cv2.CAP_GSTREAMER, 0, fps, (width, height), True
    )

while True:
    frame = np.random.randint(255, size=(height, width, 3), dtype=np.uint8)

    out_send.write(frame)

    time.sleep(0.05)

This starts the pipeline but I am unable to receive using the following pipeline. The streaming wont begin, just hangs.

gst-launch-1.0 udpsrc port=5000 ! application/x-rtp,media=video,payload=26,clock-rate=90000,encoding-name=JPEG ! rtpjpegdepay ! jpegdec ! videoconvert ! queue ! xvimagesink

However if the frame is captured from webcam as below

cap_send = cv2.VideoCapture(0)

fps = int(cap_send.get(cv2.CAP_PROP_FPS))
width = int(cap_send.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(cap_send.get(cv2.CAP_PROP_FRAME_HEIGHT))

while True:
    ret, frame = cap_send.read()

    if not ret:
        break

    out_send.write(frame)

Then the receiving pipeline starts and streams without any issues.

In both cases I see the following message on the receiving end

Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock

Edit:

I ran the receiving pipeline with GST_DEBUG=3 and see the following output

gst_rtp_jpeg_depay_process:<rtpjpegdepay0> discarding data packets received when we have no header

This only happens if the sending pipeline uses raw images / numpy arrays as above.

What am I doing incorrect here. Very new to gstreamer and need some help

Goal: To stream images generated in opencv, asynchronously.