'Cant use libcurl async

Using curl_easy, I download files wrapped in std::async. 3 asynchronous requests have been created, but a problem arises when one of the requests stumbles upon a large file and starts downloading it, the whole program hangs from this. It turns out that asynchronous requests are completely non-asynchronous.

using namespace std;

int allGameFileCount = 10000; // The count of files are written here
string allGameFiles[10000] = {}; // The file names are written here

bool downloadFile(const char* filename, const char* out)
{
    CURL* curl;
    curl = curl_easy_init();
    if (curl)
    {
        FILE* fp;
        fp = fopen(out, "wb");
        curl_easy_setopt(curl, CURLOPT_URL, url);
        curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1L);
        curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, curlWriteToFile);
        curl_easy_setopt(curl, CURLOPT_WRITEDATA, fp);
        CURLcode result = curl_easy_perform(curl);

        long httpStatusCode = 0;
        curl_easy_getinfo(curl, CURLINFO_RESPONSE_CODE, &httpStatusCode);
        if (result == CURLE_OK && httpStatusCode == 200)
        {
            // Success
            curl_easy_cleanup(curl);
            fclose(fp);
            return true;
        }
        fclose(fp);
    }
    curl_easy_cleanup(curl);
    return false;
}

int main() {
       future<void> f1 = async(launch::async, [] {
        if (downloadGameFile(allGameFiles[allGameFileCount].c_str(), allGameFiles[allGameFileCount].c_str()))
        {
          allGameFiles[allGameFileCount] = "";
          allGameFileCount--;
        }});

       future<void> f2 = async(launch::async, [] {
        if (downloadGameFile(allGameFiles[allGameFileCount-1].c_str(), allGameFiles[allGameFileCount-1].c_str()))
        {
          allGameFiles[allGameFileCount-1] = "";
          allGameFileCount--;
        }});

       future<void> f3 = async(launch::async, [] {
        if (downloadGameFile(allGameFiles[allGameFileCount-2].c_str(), allGameFiles[allGameFileCount-2].c_str()))
        {
          allGameFiles[allGameFileCount-2] = "";
          allGameFileCount--;
        }});
}


Solution 1:[1]

First of all, you are accessing the allGameFiles array out or range:

int allGameFileCount = 10000; // The count of files are written here
string allGameFiles[10000] = {}; // The file names are written here
// ...
allGameFiles[allGameFileCount]

That is an Undefined Behavior, and that itself may cause an issue. You even assing a new value to the element outside of the range: one more reason to experience UB.

Next, you are trying to get c_str() out of an invalid object, and that is another reason of UB.

Finally you are assigning an empty string to some items in the array, but the index is being changed from different threads. That means that you may assign the value to the same element from different threads, and that is not safe.

One more reason for UB is the decrement of the same variable from different threads:

allGameFileCount--;

You should use atomics instead.

There may be other issues, but fix these ones first, and your problem would probably disappear.

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1 Dmitry Kuzminov