Wait for ssh connection to be available and run task with same ssh connection afterwards

Question

The case:

Multiple servers try to connect a server with delegate_to and execute task simultaneously where ssh connection is limited with /etc/security/limits.conf and /etc/ssh/sshd_config and some of the servers fail due to limitation.

e.g: Server X has limits of 3 ssh connection at the same time. Servers 1,2,3,4,5 try to execute a task on Server X at the same time. They get error: too many logins for <remote_user>.

What I have tried:

1. Adding ssh retries to both ansible.cfg and delegate_to vars but it does not work because ssh retries for only return code of 255 where too many logins give 254 and doesn't try retry.
2. Adding wait_for_connection as a pre-task. This waits for possible connections but since this task and next task are independent, the ssh connection is independent too so it fails if another connection comes in at the same time after wait_for_connection.

Looking for any possible solution for this.

Notes: Every server runs its own ansible task. Ansible version is 2.9.9 on servers.