'Django's infinite streaming response logs 500 in apache logs

I have a Django+Apache server, and there is a view with infinite streaming response

def my_view(request):
    try:
        return StreamingHttpResponse(map(
            lambda x: f"{dumps(x)}\n", 
            data_stream(...)  # yields dicts forever every couple of seconds
        ))
    except Exception as e:
        print_exc()
        return HttpResponse(dumps({
            "success": False,
            "reason": ERROR_WITH_CLASSNAME.format(e.__class__.__name__)
        }), status=500, content_type="application/json")

When client closes the connection to the server, there is no cleanup to be done. data_stream will yield one more message which won't get delivered. No harm done if that message is yielded and not received as there are no side-effects. Overhead from processing that extra message is negligible on our end.

However, after that last message fails to deliver, apache logs 500 response code (100% of requests). It's not getting caught by except block, because print_exc doesn't get called (no entries in error log), so I'm guessing this is apache failing to deliver the response from django and switching to 500 itself.

These 500 errors are triggering false positive alerts in our monitoring system and it's difficult to differentiate an error due to connection exit vs an error in the data_stream logic.

Can I override this behavior to log a different status code in the case of a client disconnect?

djangoapache


Solution 1:[1]

From what I understand about the StreamingHttpResponse function is that any exceptions raised inside it are not propagated further. This has to do with how WSGI server works. If you start handling an exception and steal the control, the server will not be able to to finish the HTTP response. So the error is handled by the server and printed in the terminal. If you attach the debugger to this and see how the exception is handled you will be able to find a line in wsgiref/handlers.py where your exception is absorbed and taken care of.

I think in this file- https://github.com/python/cpython/blob/main/Lib/wsgiref/handlers.py

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1 Vikram Gupta

Related Questions