Passenger issue - boost::thread_resource_error: Resource temporarily unavailable

Question

i am runing a virtual server with "Plesk Obsidian Version 18.0.41" on Ubuntu 20.04.4 LTS.

I have passenger installed to run OpenProject.

Somehow the performance is not so good and i get after some time a lot of 500 http error codes from the api calls of openproject.

After checking several logs i´ve found passenger errors in the apache log

[ N 2022-03-08 00:00:43.9974 344600/T1 age/Wat/WatchdogMain.cpp:1373 ]: Starting Passenger watchdog...
[ N 2022-03-08 00:00:44.0636 344603/T1 age/Cor/CoreMain.cpp:1340 ]: Starting Passenger core...
[ N 2022-03-08 00:00:44.0638 344603/T1 age/Cor/CoreMain.cpp:256 ]: Passenger core running in multi-application mode.
[ N 2022-03-08 00:00:44.0980 344603/T1 age/Cor/CoreMain.cpp:1015 ]: Passenger core online, PID 344603
[Tue Mar 08 00:00:44.161230 2022] [ssl:warn] [pid 166471:tid 140525254335552] AH01909: default:443:0 server certificate does NOT include an ID which matches the server name
[Tue Mar 08 00:00:44.161808 2022] [mpm_event:notice] [pid 166471:tid 140525254335552] AH00489: Apache/2.4.41 (Ubuntu) mod_fcgid/2.3.9 Phusion_Passenger/6.0.12 OpenSSL/1.1.1f configured -- resuming normal operations
[Tue Mar 08 00:00:44.161833 2022] [core:notice] [pid 166471:tid 140525254335552] AH00094: Command line: '/usr/sbin/apache2'
[ N 2022-03-08 00:00:50.0828 344603/T7 age/Cor/SecurityUpdateChecker.h:519 ]: Security update check: no update found (next check in 24 hours)
[Tue Mar 08 03:56:52.557626 2022] [cgi:error] [pid 344629:tid 140525046802176] [client x.x.x.x:y] AH02811: script not found or unable to stat: /var/www/vhosts/default/cgi-binindex2.asp, referer: http://x.x.x.x:
80/cgi-bin/index2.asp
[Tue Mar 08 06:13:21.410960 2022] [core:error] [pid 344629:tid 140525080372992] [client x.x.x.x:y] AH00126: Invalid URI in request GET /cgi-bin/.%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/etc/host
s HTTP/1.1
[Tue Mar 08 10:53:32.314469 2022] [core:error] [pid 344629:tid 140525155907328] [client x.x.x.xy:y] AH00126: Invalid URI in request POST /cgi-bin/.%2e/.%2e/.%2e/.%2e/bin/sh HTTP/1.1
App 370966 output: [passenger_native_support.so] not compiling because PASSENGER_COMPILE_NATIVE_SUPPORT_BINARY=0
App 370966 output: [passenger_native_support.so] not downloading because PASSENGER_DOWNLOAD_NATIVE_SUPPORT_BINARY=0
App 370966 output: [passenger_native_support.so] will not be used (can't compile or download)
App 370966 output: --> Passenger will still operate normally.
App 372114 output: [passenger_native_support.so] not compiling because PASSENGER_COMPILE_NATIVE_SUPPORT_BINARY=0
App 372114 output: [passenger_native_support.so] not downloading because PASSENGER_DOWNLOAD_NATIVE_SUPPORT_BINARY=0
App 372114 output: [passenger_native_support.so] will not be used (can't compile or download)
App 372114 output: --> Passenger will still operate normally.
terminate called after throwing an instance of 'Passenger::SystemException'
what():  Cannot fork() a new process: Resource temporarily unavailable (errno=11)
ERROR: cannot fork a process for executing 'tee'

[ pid=344603, timestamp=1646761144 ] Process aborted! signo=SIGABRT(6), reason=SI_TKILL, signal sent by PID 344603 with UID 0, si_addr=0x5421b, randomSeed=1646694044
[ pid=344603 ] Crash log files will be dumped to /var/tmp/passenger-crash-log.1646761144.CRnqi5 <--- ******* LOOK HERE FOR DETAILS!!! *******
[ pid=344603 ] Could not fork a child process for dumping diagnostics: fork() failed with errno=11
[ W 2022-03-08 18:39:04.9338 344600/T4 age/Wat/AgentWatcher.cpp:94 ]: Passenger core (pid=344603) crashed with signal SIGABRT, restarting it...
[ N 2022-03-08 18:39:05.9515 372279/T1 age/Cor/CoreMain.cpp:1340 ]: Starting Passenger core...
[ N 2022-03-08 18:39:05.9519 372279/T1 age/Cor/CoreMain.cpp:256 ]: Passenger core running in multi-application mode.
[ N 2022-03-08 18:39:06.0103 372279/T1 age/Cor/CoreMain.cpp:1015 ]: Passenger core online, PID 372279
App 372338 output: [passenger_native_support.so] not compiling because PASSENGER_COMPILE_NATIVE_SUPPORT_BINARY=0
App 372338 output: [passenger_native_support.so] not downloading because PASSENGER_DOWNLOAD_NATIVE_SUPPORT_BINARY=0
App 372338 output: [passenger_native_support.so] will not be used (can't compile or download)
App 372338 output: --> Passenger will still operate normally.
[ N 2022-03-08 18:39:08.1801 372279/T5 age/Cor/SecurityUpdateChecker.h:519 ]: Security update check: no update found (next check in 24 hours)
[ E 2022-03-08 18:39:35.1610 372279/Tn age/Cor/App/Implementation.cpp:221 ]: Could not spawn process for application /opt/openproject: An internal error occurred while spawning an application process: boost::thread_resource_
error: Resource temporarily unavailable
Error ID: d59f2d1e
Error details saved to: /tmp/passenger-error-FuTlIc.html

[ E 2022-03-08 18:39:35.1791 372279/Tf age/Cor/Con/CheckoutSession.cpp:281 ]: [Client 4-1] Cannot checkout session because a spawning error occurred. The identifier of the error is d59f2d1e. Please see earlier logs for detai
ls about the error.
[ E 2022-03-08 18:39:35.1792 372279/T9 age/Cor/Con/CheckoutSession.cpp:281 ]: [Client 1-2] Cannot checkout session because a spawning error occurred. The identifier of the error is d59f2d1e. Please see earlier logs for detai
ls about the error.

in the html log of this line "

Crash log files will be dumped to /var/tmp/passenger-crash-log.1646761144.CRnqi5 <--- ******* LOOK HERE FOR DETAILS!!! *******

"

the output is:

Here the detail resource of my server

Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz (6 core(s))
16.00 GB of RAM (13.88 GB free)

lscpu gives me the follow:

Architektur: x86_64
CPU Operationsmodus: 32-bit, 64-bit
Byte-Reihenfolge: Little Endian
Adressgrößen: 46 bits physical, 48 bits virtual
CPU(s): 6
Liste der Online-CPU(s): 0-5
Thread(s) pro Kern: 1
Kern(e) pro Socket: 6
Sockel: 1
Anbieterkennung: GenuineIntel
Prozessorfamilie: 6
Modell: 63
Modellname: Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz
Stepping: 2
CPU MHz: 1999.918
BogoMIPS: 4993.74
Virtualisierung: VT-x
Hypervisor-Anbieter: Parallels
Virtualisierungstyp: Container
L1d Cache: 192 KiB
L1i Cache: 192 KiB
L2 Cache: 1,5 MiB
L3 Cache: 30 MiB
Markierungen: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mc
a cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss
ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arc
h_perfmon pebs bts rep_good nopl xtopology nonstop_tsc
aperfmperf eagerfpu cpuid_faulting pni pclmulqdq dtes64
monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtp
r pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_d
eadline_timer aes xsave avx f16c rdrand lahf_lm abm epb
invpcid_single intel_ppin ssbd ibrs ibpb stibp tpr_sha
dow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1
avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_o
ccup_llc dtherm ida arat pln pts md_clear spec_ctrl int
el_stibp flush_l1d

I have read this https://www.phusionpassenger.com/library/config/nginx/optimization/ and did some changes in apache

SetEnv PASSENGER_MAX_POOL_SIZE 16

SetEnv PASSENGER_FORCE_MAX_CONCURRENT_REQUESTS_PER_PROCESS 0

SetEnv MaxConnectionsPerChild 0

SetEnv MaxSpareThreads 2500

SetEnv MaxRequestWorkers 2500

SetEnv PassengerStartTimeout 180

This time the issue also appears but later ;-)

Before it was after e.g. 1 min browsing now after 2 min

Please let me know if you need more details to help me out.

Thanks in advance for your feedback.

Best regards

Ming