'How to get a reference to previous event for a particular grouping using ksqlDB or kafka streams?
I have a stream of events called transactions. Over time, many transactions are committed by a single address and there are many addresses.
For every transaction a particular address make, I need a way to see the previous transaction the address made. Essentially, I need to create a singularly linked list of the transactions for a particular address.
Using the aggregate functions available in ksqlDB at the moment, this doesn't seem possible (however, I'd love to be proven wrong). Also, without multi-column joins, it becomes prohibitively expensive to join the incoming transaction stream with a table of previous transactions as tx_with_prev grows exponentially with the number of transactions:
with tx_with_prev as (
select tx_stream_id
, tx_id
from tx_stream
left join tx
on tx_stream.address = tx.address
)
select tx_stream_id
, max(tx_id) as tx_id
from tx_with_prev
where tx_stream_id > tx_id
group by tx_stream_id
Wondering if there's any other way to do this in ksqlDB or if there is a way to do it with kafka streams which I am completely unfamiliar with.
Any input is appreciated!
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
