'Kafka Stream - lookup by customer name and related info from the topics

Kafka Stream lookup customer by value architecture

From the above problem statement diagram ,the requirement is to fetch the attr1 and attr2 based on the incoming message containing customer: "XYZ".

customer
-------
customer_id,
customer_name

customer_config
---------------
customer_id,
attr1,
attr2,

cust_agg
--------
customer_id,
customer_name,
attr1,
attr2,

**since i use debezium and while cust rec delete,we dont get old value
 null means tombstone.
 so i am using custagggobj to get prev value,and after rekey,i will 
 push to same topic cust-agg-topic ,is it a good way to code.i think that new key and value will be synced to all instances of the app. since any cust_name falls in specific partitions, consequent updates are reflected eventually.**    

cust globalstore = fetch from customer topic 
custcfg globalstore=fetch from customer-config topic9
custagg global store= fetch from cust-agg-topic topic  // fetch    from the topic (1) 

// kstream tocustomeraggtopic = 
transform(custgstore,custcfgstore,custaggstore);
{


custobj= ctx.get(custgstore) # customer_id is key 
custcfgobj =ctx.get(custcfgstore) # customer_id is key 
custagggobj = ctx.get(custagggstore) # customer_name is key 

custaggkey = custagg(cust_name);  # use cust name as key
custaggvalue = custagg(cust,custcfg);
return KeyValue.pair(custaggkey, custaggvalue );
} 

tocustomeraggtopic.to("cust-agg-topic")  // update to the same topic (1)


Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source