gRPC streaming call which takes longer than 2 minutes is killed by hardware (routers, etc.) in between client and server

Question

Grpc.Net client:

• a gRpc client sends large amount of data to a gRpc server
• after the gRpc server receives the data from the client, the http2 channel becomes idle (but is open) until the server returns the response to the client
• the gRpc server receives the data and starts processing it. If the data processing takes longer than 2 minutes (which is the default idle timeout for http calls) then the response never reaches the client because the channel is actually disconnected, but the client does not know this because it was shutdown by other hardware in between due to long idle time.

Solution:

• when the channel is created at the gRpc client side, it must have a httpClient set on it
• the httpClient must be instantiated from a socketsHttpHandler with the following properties set (PooledConnectionIdleTimeout, PooledConnectionLifetime, KeepAlivePingPolicy, KeepAlivePingTimeout, KeepAlivePingDelay)

Code snipped:

SocketsHttpHandler socketsHttpHandler = new SocketsHttpHandler()
{
    PooledConnectionIdleTimeout = TimeSpan.FromMinutes(180),
    PooledConnectionLifetime = TimeSpan.FromMinutes(180),
    KeepAlivePingPolicy = HttpKeepAlivePingPolicy.Always,
    KeepAlivePingTimeout = TimeSpan.FromSeconds(90),
    KeepAlivePingDelay = TimeSpan.FromSeconds(90)
};

socketsHttpHandler.SslOptions.RemoteCertificateValidationCallback = (sender, cert, chain, sslPolicyErrors) => { return true; };

HttpClient httpClient = new HttpClient(socketsHttpHandler);
httpClient.Timeout = TimeSpan.FromMinutes(180);

var channel = GrpcChannel.ForAddress(_agentServerURL, new GrpcChannelOptions
{
    Credentials = ChannelCredentials.Create(new SslCredentials(), credentials),
    MaxReceiveMessageSize = null,
    MaxSendMessageSize = null,
    MaxRetryAttempts = null,
    MaxRetryBufferPerCallSize = null,
    MaxRetryBufferSize = null,
    HttpClient = httpClient
});

Answer 1

this is what I needed. I had this problem for months now, but my only solution was to decrease the volume of data.