IBasicVideoEffect with NV12 Subtype

Question

I'm writing UWP app that give's user an opportunity to add video effect's on video stream receiving by some camera. I used C# MediaFrameReader to send SoftwareBitmap in C++/CX OpenCV to add effects. After that I noticed that C# has IBasicVideoEffect interface that can add effects directly to the CaptureElement. (https://docs.microsoft.com/en-us/windows/uwp/audio-video-camera/custom-video-effects)

But according the MSDN, IBasicVideEffect implementation works good only with ARGB32 subtype. Unfortunately, my camera has only NV12 subtype. Is there a way to convert context.OutputFrame.SoftwareBitmap to ARGB32 format and then show it on the screen without convering it back to NV12?

Answer 1

I founded the solution. Provided article says that there're tho ways of implementing video effects. Second one is Win2D.uwp with IDirect3DDevice. That's worked for me without any convertations from NV12. Moreover, I've reached performance by using GPU resources for it, as MSDN say's.

My code sample:

public async Task<bool> TryStartAsync()
    if (Ready == false)
    {
        return false;
    }
    if (_effectDefinition is null)
    {
        _effectDefinition = new VideoEffectDefinition(
            typeof(VideoEffectProcessor).FullName
        );
    }
    var effect = await Capture.AddVideoEffectAsync(
        _effectDefinition, 
        MediaStreamType.VideoPreview
    );
    effect.SetProperties(Settings);

    Destination.Source = Capture;
    await Capture.StartPreviewAsync();
    InProcess = true;
    return true;
}