'How to handle source topic deletion in Mirror Maker 2?
When I try to delete a topic in the source cluster that MM2 is trying to replicate, it starts throwing the below error continuously. While this is expected, the error doesn't stop and continues forever causing huge log files on my system. Is there a way to have MM2 handle source topic deletion gracefully?
[2022-05-12 14:42:57,473] WARN [Consumer clientId=consumer-4, groupId=null] Received unknown topic or partition error in fetch for partition sourcetopic-0 (org.apache.kafka.clients.consumer.internals.Fetcher:1250)
PS: I am running MM2 in dedicated cluster mode
Solution 1:[1]
You would need to write some api that will update the MirrorMaker without the deleted topic after the topic gets deleted and restart the consumer (happens automatically when you POST an update to the Connect API)
There's no way for it to know a topic should be consumed or not, as it has a static configuration
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|---|
| Solution 1 | OneCricketeer |