'Firestore - INTERNAL: Received unexpected EOS on empty DATA frame from server

I have two snapshot listeners added to different collection groups and both of these listeners work as expected until after some time they randomly get "io.grpc.StatusRuntimeException: INTERNAL: Received unexpected EOS on empty DATA frame from server" and the listeners are removed. Both of the listeners receive this exception at the same time. It is also worth noting that there are no changes to the collection groups when the exceptions occur. I could add the listeners back when the exception occurs but that would require receiving the entirety of the collection groups again which could cause a lot of overhead. Here is the full stack trace:

com.google.cloud.firestore.FirestoreException: com.google.api.gax.rpc.InternalException: io.grpc.StatusRuntimeException: INTERNAL: Received unexpected EOS on empty DATA frame from server
    at com.google.cloud.firestore.FirestoreException.forApiException(FirestoreException.java:104) ~[google-cloud-firestore-2.6.1.jar:2.6.1]
    at com.google.cloud.firestore.Watch$2.run(Watch.java:345) [google-cloud-firestore-2.6.1.jar:2.6.1]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) [?:?]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) [?:?]
    at java.lang.Thread.run(Thread.java:831) [?:?]
Caused by: com.google.api.gax.rpc.ApiException: com.google.api.gax.rpc.InternalException: io.grpc.StatusRuntimeException: INTERNAL: Received unexpected EOS on empty DATA frame from server
    at com.google.cloud.firestore.Watch$2.run(Watch.java:348) ~[google-cloud-firestore-2.6.1.jar:2.6.1]
    ... 3 more
Caused by: com.google.api.gax.rpc.InternalException: io.grpc.StatusRuntimeException: INTERNAL: Received unexpected EOS on empty DATA frame from server
    at com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:67) ~[gax-1.66.0.jar:1.66.0]
    at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:72) ~[gax-grpc-1.66.0.jar:1.66.0]
    at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:60) ~[gax-grpc-1.66.0.jar:1.66.0]
    at com.google.api.gax.grpc.ExceptionResponseObserver.onErrorImpl(ExceptionResponseObserver.java:82) ~[gax-grpc-1.66.0.jar:1.66.0]
    at com.google.api.gax.rpc.StateCheckingResponseObserver.onError(StateCheckingResponseObserver.java:86) ~[gax-1.66.0.jar:1.66.0]
    at com.google.api.gax.grpc.GrpcDirectStreamController$ResponseObserverAdapter.onClose(GrpcDirectStreamController.java:149) ~[gax-grpc-1.66.0.jar:1.66.0]
    at io.grpc.internal.DelayedClientCall$DelayedListener$3.run(DelayedClientCall.java:463) ~[grpc-core-1.39.0.jar:1.39.0]
    at io.grpc.internal.DelayedClientCall$DelayedListener.delayOrExecute(DelayedClientCall.java:427) ~[grpc-core-1.39.0.jar:1.39.0]
    at io.grpc.internal.DelayedClientCall$DelayedListener.onClose(DelayedClientCall.java:460) ~[grpc-core-1.39.0.jar:1.39.0]
    at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:553) ~[grpc-core-1.39.0.jar:1.39.0]
    at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:68) ~[grpc-core-1.39.0.jar:1.39.0]
    at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:739) ~[grpc-core-1.39.0.jar:1.39.0]
    at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:718) ~[grpc-core-1.39.0.jar:1.39.0]
    at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37) ~[grpc-core-1.39.0.jar:1.39.0]
    at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123) ~[grpc-core-1.39.0.jar:1.39.0]
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?]
    at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) ~[?:?]
    ... 3 more
Caused by: io.grpc.StatusRuntimeException: INTERNAL: Received unexpected EOS on empty DATA frame from server
    at io.grpc.Status.asRuntimeException(Status.java:535) ~[grpc-api-1.39.0.jar:1.39.0]
    at com.google.api.gax.grpc.GrpcDirectStreamController$ResponseObserverAdapter.onClose(GrpcDirectStreamController.java:149) ~[gax-grpc-1.66.0.jar:1.66.0]
    at io.grpc.internal.DelayedClientCall$DelayedListener$3.run(DelayedClientCall.java:463) ~[grpc-core-1.39.0.jar:1.39.0]
    at io.grpc.internal.DelayedClientCall$DelayedListener.delayOrExecute(DelayedClientCall.java:427) ~[grpc-core-1.39.0.jar:1.39.0]
    at io.grpc.internal.DelayedClientCall$DelayedListener.onClose(DelayedClientCall.java:460) ~[grpc-core-1.39.0.jar:1.39.0]
    at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:553) ~[grpc-core-1.39.0.jar:1.39.0]
    at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:68) ~[grpc-core-1.39.0.jar:1.39.0]
    at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:739) ~[grpc-core-1.39.0.jar:1.39.0]
    at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:718) ~[grpc-core-1.39.0.jar:1.39.0]
    at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37) ~[grpc-core-1.39.0.jar:1.39.0]
    at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123) ~[grpc-core-1.39.0.jar:1.39.0]
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?]
    at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) ~[?:?]
    ... 3 more

Here is the snapshot listeners:

private final ExecutorService listenerExecutor = Executors.newSingleThreadExecutor(r -> {
    Thread thread = new Thread(r, "firestore-listener");
    if (thread.isDaemon()) {
        thread.setDaemon(false);
    }
    if (thread.getPriority() != Thread.NORM_PRIORITY) {
        thread.setPriority(Thread.NORM_PRIORITY);
    }
    return thread;
});

public void init() {
    Firestore db = FirestoreClient.getFirestore();
    db.collectionGroup("info").addSnapshotListener(listenerExecutor, (snapshots, e) -> {
        if (e != null) {
            logger.error("Error in info snapshot listener", e);
            return;
        }

        ...
    });

    db.collectionGroup("entries").addSnapshotListener(listenerExecutor, (snapshots, e) -> {
        if (e != null) {
            logger.error("Error in entries snapshot listener", e);
            return;
        }

        ...
    });
}


Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source