'Nginx load balancer with limit_conn and burst
I currently have multiple backend-servers that are responsible for handling API-requests and one Nginx load balancer that redirects requests to these backend-servers. Since some of these API-requests can be very compute intensive and some clients send a lot of requests at the same time, I want to limit the amount of requests that are sent from the load balancer to the backend-servers per client IP (such that maximum 2 requests for a client are handled simultaneously by the backend). It seems like limit_conn can be used to achieve this but with one major limitation. I don't want the load-balancer to respond with an HTTP error in case the client sends 5 requests at once (instead of the maximum 2), but I want the load balancer to keep track of these requests and place them in a queue.
What I really need is a combination of limit_conn and the burst parameter from limit_req. Is anything like this possible? I can't seem to find it.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|