'Streamsets pipeline didn't finish if kafka consumer having 0 messages in topic
I have developed a streamsets pipeline which using KAFKA consumer as origin.My pipeline is working fine if Kafka consumer having message in it.But kafka consumer have 0 message in it the my pipeline went into loop and running contineously and didn't finish.
I need to finish my pipeline if kafka consumer having zero messages in his topic.
Solution 1:[1]
By default it is streaming application's nature to keep checking for messages. But a similar approach to what you want to achieve has been answered-here.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|---|
| Solution 1 | Learner |