'Kafka state store range query Key formation
I am trying to query Kafka State Store using range query. My key format is satrtTimeInMillis|endTimeInMillis|someUniqueIdInNumber and using String to store records.
I want to execute range query like
store.range("fromSomeStartTimeInMillis","toCurrentTimeInMillis")
OR
store.range("fromSomeEndTimeInMIllis","toCurrentTimeInMillis")
My expectation is, if I execute query based on startTime, it should only return all records between from,to startTime, same likewise should hold for endTime too.
e.g
<1|2|9>, <2|1|8>,<3|2|7>,<100|2|4>,<100|5|11>
(say stored keys)
Query : i) store.range("1|","3|") --startTime-- It should return <1|2|9>, <2|1|8>,<3|2|7>
ii) store.range("|1","2") --endTime-- It should return <1|2|9>, <2|1|8>,<3|2|7>,<100|2|4>
I have been trying like this but results are not expected, also tried with Custom Serde Class to store records, but not any success.
Kindly suggest String Query Format for range query-- from and to for this kind of situation or even Custom Key Class with from and to fields.
Or is there any other way to handle this type of situation where I want to divide work across nodes, but there is no token service available to distribute load. Kafka seems only way to distribute load by partitioning data.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
