glm::lookAt returns matrix with nan elements

Question

I want to create a view matrix for a camera which perpendicularly look at the ground:

glm::mat4 matrix = glm::lookAt(glm::vec3(0.0f, 1.0f, 0.0f), glm::vec3(0.0f, 0.0f, 0.0f), glm::vec3(0.0f, 1.0f, 0.0f));

The last argument is the global up vector so everything seems to be correct but I get following matirx:

-nan    -nan    -0  0   
-nan    -nan     1  0   
-nan    -nan    -0  0   
 nan     nan    -1  1

I guess that I get nan because a look at vector is parallel to up vector, but how can I build a correct view matrix using glm::lookAt function.

Answer 1

In lookAt it is impossible to have the viewing direction and the up-vector looking in the same direction. If you want to have a camera that is looking along the negative y-axis, you'll have to adjust the up-vector, for example to [0,0,1]. The direction one specifies in the up-vector controls how the camera is rotated around the view axis.

Answer 2

I ran across this same problem of NaNs in the matrix returned by glm::lookAt() yesterday and have concocted what I think is a workaround. This seems to work for me for the particular problem of the UP vector being vec3(0.0f, 1.0f, 0.0f), which seems to be a common use case.

My Vulkan code looks like this:

struct UniformBufferObject {
    alignas(16) glm::mat4 model;
    alignas(16) glm::mat4 view;
    alignas(16) glm::mat4 proj;
};
...
        UniformBufferObject ubo{};
...
        glm::vec3 cameraPos = glm::vec3(0.0f, 2.0f, 0.0f);
        ubo.view = glm::lookAt(cameraPos, glm::vec3(0.0f, 0.0f, 0.0f), glm::vec3(0.0f, 1.0f, 0.0f));

        // if the direction vector from the camera to the point being observed ends up being parallel to the UP vector
        // glm::lookAt() returns a mat4 with NaNs in it. to workaround this, look for NaNs in ubo.view
        int view_contains_nan = 0;

        for (int col = 0; (col < 4) && !view_contains_nan; ++col) {
            for (int row = 0; (row < 4) && !view_contains_nan; ++row) {
                if (std::fpclassify(ubo.view[col][row]) == FP_NAN) {
                    view_contains_nan = 1;
                }
            }
        }

        // if we ended up with NaNs, the workaround ubo.view that seems to work depends on the sign of the camera position Y
        if (view_contains_nan) {
            std::cout << "view contains NaN"  << std::endl;
            if (cameraPos.y >= 0.0f) {
                ubo.view = glm::mat4(   -0.0f, -1.0f,  0.0f,        0.0f,
                             0.0f,  0.0f,  1.0f,        0.0f,
                            -1.0f,  0.0f, -0.0f,        0.0f,
                            -0.0f, -0.0f, -cameraPos.y, 1.0f);
            } else {
                ubo.view = glm::mat4(    0.0f,  1.0f,  0.0f,        0.0f,
                             0.0f,  0.0f, -1.0f,        0.0f,
                            -1.0f,  0.0f, -0.0f,        0.0f,
                            -0.0f,  -0.0f, cameraPos.y, 1.0f);
            }
        }

Hopefully it works for you too, though I suppose it would be nice if glm::lookAt() could be fixed to not return matrices with NaNs in it.