dlib c++ : slow convergence

Question

I am trying to use dlib for a real time application. I am using find_min_global in dlib to optimize a function with 2 variables. The problem is that it is taking too long to converge. If I use find_min_global(optimizationFunction, lowValue, highValue, std::chrono::milliseconds(10)), I dont get any optmization at all. The returned values are the same as input values. I do get far better results with find_min_global(optimizationFunction, lowValue, highValue, std::chrono::milliseconds(100)) but I cannot afford to spend 100 milisecs on this problem. Is there any way i can speed this up or any other function/library that can help me achieve this?

minimizeSSE()
{
    auto optimize = [&](float x, float y)
    {
        float totalError = 0;
        for(int i=0; i<N; ++i)
        {
            totalError += ((a[i] - x)*(a[i]- x) + (b[i] - y)*(b[i]- y));
        }
        return totalError;
    }

    column_vector lowerBound = {0, 0};
    column_vector upperBound = {100, 100};

    auto res = find_min_global(optimize, lowerBound, upperBound, std::chrono::milliseconds(100));
}

Here, column_vector is typedef matrix<double,0,1> column_vector; The variable res has optimized values of x and y. If I run the function minimizeSSE() , it gives me optimal values but if I run the function for 10 milisecs, I dont get any optimization.

The value of N in the loop count, is not more than a few 100s. I dont have millions of values over which I have to optimize x and y, at max a few 100s.