'How does Debezium guarantee all events within a single topic 'totally-ordered'?
As I read from Debezium's FAQs, it is said that:
Most connectors will record all events for a single database table to a single topic. Additionally, all events within a topic are totally-ordered, meaning that the order of all of those events will be maintained.
How are events for a database organized?
However, AFAIK Apache Kafka only has ordering guarantees within a single partition. So if I expect the events in a topic to be ordered, I have to set that topic having only one partition, which sacrifices the throughput of Kafka, otherwise with other mechanism. But I didn't see any explanation about this in Debezium's documentation.
My question is, how does Debezium implement the ordering guarantees within one topic? Or which module of the source code should I study to find out the detailed implementation of this feature?
Solution 1:[1]
to quote the answer here:
... Kafka Connect’s producer will use the default partitioning logic that computes the partition using a consistent hash of the message key, which in Debezium’s case is a struct containing the affected row’s primary/unique key...
So if the concern is that the same row/document should not be read out of order, then the concern is ruled out because the PK will always send the the event to the same partition
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|---|
| Solution 1 | cyberjar09 |