Is there any example of Spring Schedule that reads Kafka topic?

Question

We're trying to read the data from Kafka at specified window time (so we have Kafka consumer), that means avoiding the data read at other times. However we're not sure how to shut down the consumer after the time period is expired. I wonder if there are any example of how to do that? Many thanks in advance for helping us.

Answer 1

I had the same use case but I wrote the scheduler specifying the max poll record for one batch and kept a counter if the counter matched the max polled record then I consider that the processing for the batch is finished as it has processed the record which it got during the one poll.

And then I am unsubscribing to the topic and closing the consumer. The next time when the scheduler will run it will again process the max poll record specified limit.

fixedDelayString fulfil the purpose that the scheduler starts after the specified time limit once the previous is finished.

@EnableScheduling
public class MessageScheduler{

        @Scheduled(initialDelayString = "${fixedInitialDelay.in.milliseconds}", fixedDelayString = "${fixedDelay.in.milliseconds}")
        public void run(){

            /*write your kafka consumer here with manual commit*/

            /*once your batch is finished processing unsubcribe and close the consumer*/
                kafkaConsumer.unsubscribe();
                kafkaConsumer.close();
        }

}