'Random HTTP/1.1 502 Bad Gateway in PHP REST API

I have written a REST API in PHP using slim micro framework.

Hosted on Apache (Event MPM) + PHP8.0-FPM on 2-vCPU and 4 GiB RAM and 10Gbps N/w bandwidth behind LoadBalancer.

Issue:

The test program is receiving random HTTP/1.1 502 Bad Gateway in response error. The frequency of the error is 10 out of 1500-2000 requests.

Apache2.conf

Timeout 600
KeepAlive On
MaxKeepAliveRequests 100
KeepAliveTimeout 5

Apache Event MPM config:

<IfModule mpm_event_module>
    ServerLimit     50
    StartServers             25
    MinSpareThreads      75
    MaxSpareThreads      250
    ThreadLimit          256
    ThreadsPerChild      64
    MaxRequestWorkers     3200
    MaxConnectionsPerChild   0
</IfModule>

PHP-FPM config:

pm.max_children=700
pm.start_servers=200
pm.min_spare_server=200
pm.max_spare_server=500

I have written a separate program in C# to stress test the REST API.

Findings:

  • The server is running on 30-40% CPU and 700-900MB of RAM usage.
  • I have checked apache access logs, and no Error 502 request log is there.
  • I have checked apache error logs, and nothing is there as well.

What could be the issue? I do not know which area to debug forward.



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source