'Purpose of Debezium's Postgres connector offsets?
I need some explanation on how Postgres' pg_logical plugin and Debeziums Postgres connector work together in the context of offsets on Debezium side.
As far as I understand when reading records from the WAL pg_logical will remove them from the replication slot. So as soon as Debezium reads a record it is gone from the slot forever. Still there is a way to "remember" the last succesffuly read record on Debezium side. This is implemented via an offset storage.
I mean it seems to make sense for the case that Postgres failed and after restart the slot contains already-consumed records and the Debezium connector needs to skip some elements, but besides that?
I thought that recovering from a failure of Debezium including loosing already read WAL will include using the externally backed-up the offsets somehow.But how? The replication slot will already have discarded the records that have been consumed before the connector failure...
Thanks a lot for enlightening me!
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
