'Assigned Offset is not availble with Kafka Cosnumer (Offset out of Range)
I am consuming from a Kafka topic with Assign Offset
KafkaUtils.createDirectStream(sparkStreamingContext, PreferConsistent, Assign[String, String](fromOffsets.keys, kafkaConsumerParams, fromOffsets))
We have Kafka consumer messages retention period as 4 days.
Suppose If I try to consume message from topic which already expired because of retention. Its throwing offset out of range exception.
So I want to check whether the assigning offset is present or not in consumer topic. If not present I'll make auto.offset.reset as earliest.
Please any one give Suggestion. How do I check assigning offset is present or not in Kafka.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
