How many bytes can be send() over tcp without ever receive(), before send() blocks -- dependent on buffer sizes?

Question

In python 3.9 I wrote a TCP server that never calls receive(). And a client that sends 1KB chunks to the server. Previously I'm setting send- and receive buffer sizes in the KB-range.

My expectation was to be able to send (send-buffer + receive-buffer) bytes before send() would block. However:

• On Windows 10: send() consistently blocks only after (2 x send-buffer + receive-buffer) bytes.
• On Raspberry Debian GNU/Linux 11 (bullseye):
  • setting buffer seizes (with setsockopt) results in twice the buffer (as reported by getsockopt).
  • send() blocks after roughly (send-buffer + 2 x receive-buffer) bytes wrt the buffer sizes set with setsockopt.

Questions: Where does the "excess" data go? How come, the implementation behave to differently?

All tests where done on the same machine (win->win, raspi->raspi) with various send/ receive buffer sizes in the range 5 - 50 KB.

Answer 1

TCP is a byte stream, there is no 1:1 relationship between sends and reads. send() copies data from the sender's buffer into a local kernel buffer, which is then transmitted to the remote peer in the background, where it is received into a kernel buffer, and finally copied by receive() into the receiver's buffer. send() will not block as long as the local kernel still has buffer space available. In the background, the sending kernel will transmit buffered data as long as the receiving kernel still has buffer space available. receive() will block only when the receiving kernel has no data available.