'Can we create ksql table from topic together with topic data?

I have a db connector, which will detect the timestamp column update on my user table on my database and push it to my kafka topic. Thus, my kafka topic will keep append with the same userid record data. And for my requirement, I always need to get the latest record of the same user.

Because of ksql does not support for order by command, I would like to create a table based on the kafka topic and together with all the data together.

Based on this, I created the user table with following command:

create table ecuser (userid varchar primary key, username varchar, idno varchar, cif varchar) with (kafka_topic = 'db2-03-', value_format='JSON');

where db2-03- is my kafka topic, and the topic having the following data:

ksql> print 'db2-03-' from beginning;
Key format: ¯\_(ツ)_/¯ - no data processed
Value format: AVRO or KAFKA_STRING
rowtime: 2022/03/16 13:39:16.980 Z, key: <null>, value: {"USERID": "1268303452638004", "USERNAME": "testing3", "IDNO": "851208023333", "CIF": "200000013"}, partition: 0
rowtime: 2022/03/16 13:39:21.984 Z, key: <null>, value: {"USERID": "1268303452638004", "USERNAME": "testing3", "IDNO": "851208023333", "CIF": "200000013"}, partition: 0
rowtime: 2022/03/16 13:39:26.989 Z, key: <null>, value: {"USERID": "1268303452638004", "USERNAME": "testing3", "IDNO": "851208023333", "CIF": "200000013"}, partition: 0
rowtime: 2022/03/16 13:39:31.993 Z, key: <null>, value: {"USERID": "1268303452638004", "USERNAME": "testing3", "IDNO": "851208023333", "CIF": "200000013"}, partition: 0
^CTopic printing ceased

After creating the ecuser table, I found that there is no data inside it. Even if I push new data into db2-03- topic, my ecuser table still remains empty.

Is my way to create the table correct?

Another question is, is my requirement suitable on this case?



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source