'Debezium Server for Azure Event hub sink send messages to multiple partition keys

I'm implementing CDC for a PostgreSQL azure database. And I want the events to be sent to azure event hub. My current plan is to use Debezium Server with the Event Hub sink to do this. However I want to enforce order of events by table. From this article I know I can do this by having a single topic with multiple partitions but only sending events from a single table to a specific partition every time.

However it seems like debezium doesn't provide a nice way to handle this. You can specify the partition key for all events to be sent to, but not dynamically per event. The only other thing I saw that could solve this is a custom sink implementation or a custom EventHubProducerClient implementation passed into the config.

What are my options for handling this? Is there another way to architect this solution so that I don't have to use partition keys? Or is a custom sink implementation going to be my best bet? Or should i just drop debezium and write a custom listener/publisher?

Context / requirements

  • typically to run debezium you need a kafka instance running, however if possible I don't want to use kafka as I'm already planning on using event hub, and it seems duplicitous, and it is another service that needs to be maintained.
  • FIFO ordering of events by table when read by consumers of the event hub
  • all logical database changes are turned into events
  • no java developers on the team so the custom (java) implementations will be a stretch to our expertise.


Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source