'Kerberized Apache Kafka consumer not creating groups
I am using version 2.13. Kafka is able to produce messages but not able to consume messages after investigation I found that consumer groups are not getting created and hence the issue. I am able to consume using partition and offset option but not with group option.
Kafka is kerberized, I am trying to produce messages using following command and this works fine
export KRB5CCNAME=/home/kafka/krb5cc;KAFKA_HEAP_OPTS="-Djava.security.krb5.conf=/etc/krb5.conf -Dsun.security.krb5.debug=true";
./kafka-console-producer.sh --producer.config ../config/producer.properties --topic kerbtopic4 --bootstrap-server kafka:6092
Consumer command this works fine but cannot read messages
export KRB5CCNAME=/home/kafka/krb5cc;KAFKA_HEAP_OPTS="-Djava.security.krb5.conf=/etc/krb5.conf -Dsun.security.krb5.debug=true";
./kafka-console-consumer.sh --consumer.config ../config/sasl.properties --topic kerbtopic4 --bootstrap-server ddr-kafka:6092 --from-beginning
Consumer command this works fine and can read messages when I give partition and offset
export KRB5CCNAME=/home/kafka/krb5cc;KAFKA_HEAP_OPTS="-Djava.security.krb5.conf=/etc/krb5.conf -Dsun.security.krb5.debug=true";
./kafka-console-consumer.sh --consumer.config ../config/sasl.properties --topic kerbtopic4 --bootstrap-server kafka:6092 --partition 0 --offset earliest
This command does not return any group
export KRB5CCNAME=/home/kafka/krb5cc;KAFKA_HEAP_OPTS="-Djava.security.krb5.conf=/etc/krb5.conf -Dsun.security.krb5.debug=true";
./kafka-consumer-groups.sh --bootstrap-server kafka:6092 --list --command-config ../config/consumer.properties
Solution 1:[1]
There was no issue with configuration. I had to re-install kafka in a different folder and that resolved the issue. So not sure what caused the issue and how did kafka setup get corrupted such that consumer was unable to create group.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
Solution | Source |
---|---|
Solution 1 | Manoj |