'Kafka offset is set to -1000
My Kafka Consumer says the following:
[TopicPartition{topic=my-topic,partition=0,offset=-1000,error=None}]
Whenever I run poll(), it returns None.
I want the offset to be set to the last consumed message. If I manually set the offset=0, anytime I re-start the consumer, all messages will be re-read -- I only want to consume messages from the last offset.
Why is my offset set to -1000?
How can I have kafka manage the offset for me instead of it always being -1000?
What does a -1000 offset mean, it seems pretty invalid.
Solution 1:[1]
I would think of 3 things:
- Could you please confirm if the consumer received messages at any point of time?
- Please try below cli command to see the lag of your consumer:
kafka-consumer-groups --bootstrap-server <> --group <consumer-group> --command-config <kafka_client.properties-path> --describe
- How are you committing the offsets in your consumer code?(check enable.auto.commit property; if true kafka client should commit periodically)
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|---|
| Solution 1 |