2 different consumer application instances with the same group id consuming messages from the same topic

Question

What happens If there are 2 different consumer application instances with the same group id consume from the same topic ?

Example : If the consumer application is developed by 2 different teams, running parallel with the same group id and reading from the same kafka topic.

Answer 1

In Kafka we have Consumer and Consumer groups.

Kafka builds on the publish-subscribe model with the advantages of a message queuing system. It achieves this with-

• the use of consumer groups
• message retention by brokers

Consumers that share the same group id are part of the same Consumer group. The Consumers from the same Consumer Group works as message queuing system, so if a message is being consumed by one of the consumers, it won't be available to be consumed by another consumer in the same group.

Consider the below picture-

Here we have two different Consumer groups, A and B, and as we can see the Consumers in the same group are not assigned the same Topic Partition.

The consumers in a group divides the topic partitions as fairly amongst themselves as possible by establishing that each partition is only consumed by a single consumer from the group.