Cassandra timeout during read query at consistency ALL

Question

we use Cassandra 3.11.1, com.datastax.oss java-driver-core 4.13.0 and Java 13.

we have multiple read micro-services which read data from Cassandra, but got this error: Cassandra timeout during read query at consistency ALL (8 responses were required but only 7 replica responded)

our queries are mostly just select by primary key, some of queries even specify to set consistency level to local_quorum, wonder why we still ran into this issue, sample queries:

QueryBuilder.selectFrom(Email.TABLE_NAME)
.all()
.whereColumn(Email.COLUMN_EMAIL_ADDRESS)
.isEqualTo(bindMarker())
.limit(bindMarker())
.build()
.setConsistencyLevel(ConsistencyLevel.LOCAL_QUORUM)



QueryBuilder.selectFrom(Account.TABLE_NAME)
.columns(
AccountDataObject.COLUMN_ACCOUNT_ID,
AccountDataObject.COLUMN_EMAIL_ADDRESS)
.whereColumn(Account.COLUMN_ACCOUNT_ID)
.isEqualTo(bindMarker())
.build()