'Apache Kafka: How to find out consumer group of a topic?

I am working on Kafka with Spark streaming in Scala. There are multiple topics from which I need to fetch messages. For that, I need to input consumer group of each topic.

I have tried following command:

bin/kafka-consumer-groups.sh --zookeeper localhost:2181 --describe --group group_name

For this, I need to mention consumer group and check whether my topic falls under this group. But I don't know the consumer group value for some topics.

So, is there any command or programmatic way of finding consumer group for a given topic?

Any help on this would be greatly appreciated. Thank you

javascalaapache-sparkcommandapache-kafka


Solution 1:[1]

You can read data from various topics by specifying the topics. You can simply create a Map of topics. Refer to the below link

https://www.tutorialspoint.com/apache_kafka/apache_kafka_consumer_group_example.htm

Solution 2:[2]

There is no Command/API provided by kafka to list consumer groups consuming from a specified topic. Instead you can use a tool developed by yahoo to manage kafka clusters : Yahoo Kafka Manager. After installing this software and pointing yahoo kafka manager to your kafka cluster, you can view consumer groups consuming from a specified topic.

Solution 3:[3]

for i in `kafka-consumer-groups --list --bootstrap-server localhost:9092`; do echo $i; (kafka-consumer-groups --describe --bootstrap-server localhost:9092 --group $i 2>&1| grep mytopic); done

It takes some time, because it checks all the topics, but tells which consumer groups use the topic.

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1 Ishan Kumar
Solution 2
Solution 3 BartBiczBo?y

Related Questions