C++ - Failed to stream to youtube with ffmpeg

Question

Well, I have an open source project similar to OBS but aimed at churches, the repository you can find here, however I'm stuck in the encoding part, avcodec_send_frame after a while just returns Failed to send frame - code: -11 err: Resource temporarily unavailable and never gets to the stage of receiving packets.

I've tried everything including changing the output format to MP4 but the application just crashes, this time I'm challenging myself to a bigger project, I thought I had solved it by changing the codec to H.264 but I was wrong

Basically I'm trying to get OpenCV images from my webcam and send them to youtube

void Broadcast::writeFrame(const cv::Mat &image)
{

    const int width = image.cols;
    const int height = image.rows;

    if(swConv == nullptr) {
        swConv = sws_getContext(width, height, AV_PIX_FMT_RGB24, codecCtx->width, codecCtx->height, codecCtx->pix_fmt, SWS_BILINEAR, nullptr, nullptr, nullptr);
        if(!swConv) {
            qCritical() << "Could not create SwContext";
            return;
        }
    }

    const int stride[4] = { static_cast<int>(image.step[0]) };
    sws_scale(swConv, &image.data, stride, 0, image.rows, frame->data, frame->linesize);
    frame->pts = frame_pts++;

    int ret = avcodec_send_frame(codecCtx, frame);
    if(ret < 0) {
        qWarning() << "Failed to send frame - code: " << ret << " err: " << av_err2str(ret);;
        return;
    }

    qDebug() << "AA " << ret;

    // Encode frames into packet
    while(ret > 0) {

        qDebug() << "BB " << ret;

        ret = avcodec_receive_packet(codecCtx, pkt);
        if (ret == AVERROR(EAGAIN) || ret == AVERROR_EOF) {
            return;
        } else if(ret < 0) {
            qCritical() << "Error reciving packets: " << av_err2str(ret);
            break;
        }

        qDebug() << "CC";

        av_write_trailer(ofctx);
        //av_packet_unref(pkt);

    }

    qDebug() << "Frames " << frame_pts;

}