Windows Desktop Duplication Api - Trying to get desired fps with fixed time step - Problem with the Timeout from the AcquireNextFrame function

Question

I'm currently trying to create a "screenshare" program that has low latency and good performance. For this I use DirectX to capture the screen and FFmpeg to encode it. And all in C++. But now I have several problems that I just can't solve.

1. My biggest problem right now is getting frames at a fixed rate. I've already tried some things, like the sleep function, but this is way too inaccurate. Another thing I tried was some approaches from this github page (It contains code aswell) and I actually got good results for accuracy but every approach got me too high CPU or too low accuracy either way. The reason for the high CPU usage was spin locking. I'm pretty stuck at this point. I have no idea how to get a frame every ~16.67ms for example.

2. My second problem has to do with the AcquireNextFrame function timing out. Let's say I want 120 FPS, then I have a delta time of 1000 / 120 = 8.33333333333 ms, but it won't actually time out when it reaches 8 ms, but actually a bit later (about 7ms later), and then I can't feed the encoder with the last frame because the timeout was longer than 8ms so the video gets slowed down. And that happens quite often because there's a timeout when nothing changes on the screen or it's just not fast enough.

Here is the code on how I'm getting the Frame

FRAME_DATA DuplicationManager::getFrame(UINT timeout) {
FRAME_DATA result;

IDXGIResource* DesktopResource = nullptr;
DXGI_OUTDUPL_FRAME_INFO FrameInfo;

HRESULT hr = m_DesktopDup->AcquireNextFrame(timeout, &FrameInfo, &DesktopResource);

if (hr == DXGI_ERROR_ACCESS_LOST) {
    result.result = RESULT_ACCESSLOST;
    m_DesktopDup->ReleaseFrame();
    initialize();
    return result;
}

if (hr == DXGI_ERROR_WAIT_TIMEOUT) {
    result.result = RESULT_TIMEOUT;
    m_DesktopDup->ReleaseFrame();
    return result;
}

if (FAILED(hr)) {
    result.result = RESULT_ERROR;
    result.error = "Failed to aquire next frame: " + std::system_category().message(hr);
    m_DesktopDup->ReleaseFrame();
    initialize();
    return result;
}

if (m_LastImage) {
    m_LastImage->Release();
    m_LastImage = nullptr;
}

hr = DesktopResource->QueryInterface(__uuidof(ID3D11Texture2D), reinterpret_cast<void**>(&m_LastImage));
DesktopResource->Release();
DesktopResource = nullptr;

if (FAILED(hr)) {
    result.result = RESULT_ERROR;
    result.error = "Failed to QI for ID3D11Texture2D from acquired IDXGIResource: " + std::system_category().message(hr);
    m_DesktopDup->ReleaseFrame();
    return result;
}

D3D11_TEXTURE2D_DESC frameDesc;
m_LastImage->GetDesc(&frameDesc);

D3D11_TEXTURE2D_DESC stagingTextureDesc;
stagingTextureDesc.Width = frameDesc.Width;
stagingTextureDesc.Height = frameDesc.Height;
stagingTextureDesc.MipLevels = frameDesc.MipLevels;
stagingTextureDesc.ArraySize = 1;
stagingTextureDesc.Format = frameDesc.Format;
stagingTextureDesc.SampleDesc = frameDesc.SampleDesc;
stagingTextureDesc.Usage = D3D11_USAGE_STAGING;
stagingTextureDesc.BindFlags = 0;
stagingTextureDesc.CPUAccessFlags = D3D11_CPU_ACCESS_READ;
stagingTextureDesc.MiscFlags = 0;

ID3D11Texture2D* stagingTexture = nullptr;

hr = m_Device->CreateTexture2D(&stagingTextureDesc, nullptr, &stagingTexture);
if (FAILED(hr)) {
    result.result = RESULT_ERROR;
    result.error = "Failed to create shared surface: " + std::system_category().message(hr);
    m_DesktopDup->ReleaseFrame();
    return result;
}

m_Context->CopyResource(stagingTexture, m_LastImage);

result = getFrameData(stagingTexture, stagingTextureDesc);

stagingTexture->Release();
m_DesktopDup->ReleaseFrame();

return result;
}

And here is the code to turn it into a Bitmap

FRAME_DATA DuplicationManager::getFrameData(ID3D11Texture2D* texture, D3D11_TEXTURE2D_DESC& textureDesc) {
FRAME_DATA result;

D3D11_MAPPED_SUBRESOURCE resourceAccess;

HRESULT hr = m_Context->Map(texture, 0, D3D11_MAP_READ, 0, &resourceAccess);

if (FAILED(hr)) {
    result.result = RESULT_ERROR;
    result.error = "Failed to get pointer to the data contained in the shared texture: " + std::system_category().message(hr);
    return result;
}

void* imgData = malloc(textureDesc.Width * textureDesc.Height * 4);

if (imgData == NULL) {
    m_Context->Unmap(texture, 0);
    result.result = RESULT_ERROR;
    result.error = "Failed to allocate memory for the frame";
    return result;
}

char* p = (char*)resourceAccess.pData;

for (uint32_t y = 0; y < textureDesc.Height; y++) {
    memcpy((char*)imgData + y * textureDesc.Width * 4, p, textureDesc.Width * 4);
    p += resourceAccess.RowPitch;
}

char* data = reinterpret_cast<char*>(imgData);
char temp;
for (uint32_t i = 0; i < textureDesc.Width * textureDesc.Height * 4; i += 4) {
    temp = data[i + 2];
    data[i + 2] = data[i];
    data[i] = temp;
}

result.result = RESULT_SUCCESS;
result.data = data;
result.width = textureDesc.Width;
result.height = textureDesc.Height;

m_Context->Unmap(texture, 0);

return result;
}

Any help or suggestion is appreciated!

Answer 1

if you are looking for a fraction of monitor's frame rate (ex. your monitor is 120 hz and you are looking for 60, 30 etc) you can use WaitForVBlank method. This waits on monitor's Vsync signal.