'Kafka Stream - lookup by customer name and related info from the topics
Kafka Stream lookup customer by value architecture
From the above problem statement diagram ,the requirement is to fetch the attr1 and attr2 based on the incoming message containing customer: "XYZ".
customer
-------
customer_id,
customer_name
customer_config
---------------
customer_id,
attr1,
attr2,
cust_agg
--------
customer_id,
customer_name,
attr1,
attr2,
**since i use debezium and while cust rec delete,we dont get old value
null means tombstone.
so i am using custagggobj to get prev value,and after rekey,i will
push to same topic cust-agg-topic ,is it a good way to code.i think that new key and value will be synced to all instances of the app. since any cust_name falls in specific partitions, consequent updates are reflected eventually.**
cust globalstore = fetch from customer topic
custcfg globalstore=fetch from customer-config topic9
custagg global store= fetch from cust-agg-topic topic // fetch from the topic (1)
// kstream tocustomeraggtopic =
transform(custgstore,custcfgstore,custaggstore);
{
custobj= ctx.get(custgstore) # customer_id is key
custcfgobj =ctx.get(custcfgstore) # customer_id is key
custagggobj = ctx.get(custagggstore) # customer_name is key
custaggkey = custagg(cust_name); # use cust name as key
custaggvalue = custagg(cust,custcfg);
return KeyValue.pair(custaggkey, custaggvalue );
}
tocustomeraggtopic.to("cust-agg-topic") // update to the same topic (1)
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
