'Random 'Connection Refused' Errors when making get requests to my kubernetes cluster

Pretty standard setup

Nginx Ingress, with an external IP When sending bulk requests (serially) to the IP about half of them end up erroring out...example here:

$ curl -vI http://my-ip
*   Trying my-ip:80...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:--  0:00:02 --:--:--     0* connect to my-ip port 80 failed: Connection refused
* Failed to connect to my-ip port 80: Connection refused
  0     0    0     0    0     0      0      0 --:--:--  0:00:02 --:--:--     0
* Closing connection 0
curl: (7) Failed to connect to my-ip port 80: Connection refused

The other half promptly return 200 status codes.

The requests with 200 status codes I can see in the nginx controller pod logs, however I don't see anything for the Connection refused requests

The nginx pod is healthy throughout the requests, with no crashes or abnormal behavior. I am calling the nginx external IP directly

nginx          ingress-nginx-1636864778-controller                  LoadBalancer   10.101.134.218   my-ip   80:30892/TCP,443:32566/TCP     74d

Not sure where to continue from here, if theses requests aren't even reaching the controller pod?



Solution 1:[1]

enter image description here

The image above shows how HTTP requests are delivered to the pod through a LoadBalancer svc. So the problem may be related to your specific node or port availibility.

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1 YwH