What is the performance difference between stream.filter instead of CQL ALLOW FILTERING?

Question

The data in my Cassandra DB table doesn't have much data right now.

However, since it is a table where data is continuously accumulated, I am interested in performance issues.

First of all, please don't think about the part where you need to redesign the table.

Think of it as a general RDBS date-based lookup. (startDate ~ endDate)

1. From Cassandra DB Apply allow filtering and force the query. This will get you exactly the data you want.

2. Query "all data" in Cassandra DB, This query only needs to be done once. (no where) After that, only the data within the desired date is extracted through the stream().filter() function.

Which method would you choose?

In general, which one has more performance issues?

Summary: You need to do about 6 methods.

1. Execute allow filtering query 6 times / Not perform stream filter

2. Execute findAll query once / Execute stream filter 6 times

Answer 1

The challenge with both options is that neither will scale. It may work with very small data sets, say less than 1000 partitions, but you will quickly find that neither will work once your tables grow.

Cassandra is designed for real-time OLTP workloads where you are retrieving a single partition for real-time applications.

For analytics workloads, you should instead use Spark with the spark-cassandra-connector because it optimises analytics queries. Cheers!