'How to fix 502 Bad Gateway Error in production(Nginx)?

When I tried to upload a big csv file of size about 600MB in my project which is hosted in the digital ocean, it tries to upload but shows 502 Bad Gateway Error (Nginx).

The application is a data conversion application.

This works fine while working locally.

sudo tail -30 /var/log/nginx/error.log

shows

[error] 132235#132235: *239 upstream prematurely closed connection while reading response header from upstream, client: client's ip , server: ip, request: "POST /submit/ HTTP/1.1", upstream: "http://unix:/run/gunicorn.sock:/submit/", host: "ip", referrer: "http://ip/"

sudo nano /etc/nginx/sites-available/myproject

shows

server {
    listen 80;
    server_name ip;
    client_max_body_size 999M;

    location = /favicon.ico { access_log off; log_not_found off; }
    location /static/ {
       alias  /root/static/;
    }
    location / {
        include proxy_params;
        proxy_pass http://unix:/run/gunicorn.sock;
    }
}

nginx.conf

user root;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;

events {
        worker_connections 768;
        # multi_accept on;
}

http {

        ##
        # Basic Settings
        ##

        sendfile on;
        tcp_nopush on;
        tcp_nodelay on;
        keepalive_timeout 65;
        types_hash_max_size 2048;
        # server_tokens off;

        # server_names_hash_bucket_size 64;
        # server_name_in_redirect off;

        include /etc/nginx/mime.types;
        default_type application/octet-stream;

I have also the javascript loader running while the conversion process takes place. How can I fix this?

Solution 1:^[1]

If you are using django 3.1 or higher you can make your file processing asynchronous this way and return a response to the user while the file conversion takes place. Your view should look something like this...

    import asyncio
    from django.http import JsonResponse
    from asgiref.sync import sync_to_async
    
    
    @sync_to_async
    def crunching_stuff(my_file):
        # do your conversion here
            
    async def upload(request):
        json_payload = {
            "message": "Your file is being converted"
        }
        my_file = request.POST.get('file')
        
        asyncio.create_task(crunching_stuff(my_file))
        return JsonResponse(json_payload)

On the front end if you use Dropzone.js your user can see the file upload progress and will get a response quicker. this is a better user experience. https://www.dropzonejs.com/

Solution 2:^[2]

502 error can be of anything. Check your Nginx error log as following

tail /var/log/nginx/error.log -f

In my case it is because the header is too big. So one had to increase the buffer size in /etc/nginx/sites-enabled/default as Chen.A had described.

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution	Source
Solution 1	Dunski
Solution 2	zawhtut

'How to fix 502 Bad Gateway Error in production(Nginx)?

Solution 1:[1]

Solution 2:[2]

Sources

Related Questions

Solution 1:^[1]

Solution 2:^[2]