'number of socket connection in Spring Kafka
I'm trying to understand how socket connections are open when connecting to Kafka using spring Kafka. I have the following code to connect to a Kafka topic named test which has 32 partitions.
application.yml
spring:
kafka:
bootstrap-servers: testbroker:9092
consumer:
enable-auto-commit: false
auto-offset-reset: earliest
spring.kafka.consumer.key-deserializer: org.apache.kafka.common.serialization.StringDeserializer
spring.kafka.consumer.value-deserializer: org.springframework.kafka.support.serializer.JsonDeserializer
group-id: test
properties:
max.poll.records: 125
partition.assignment.strategy: org.apache.kafka.clients.consumer.RoundRobinAssignor
listener:
type: single
ack-mode: batch
following java class to listen to Kafka topic
@Component
public class GenericKafkaMessageListener {
@KafkaListener(topics = "test-topic", concurrency = "32") throws Exception {
log.info("message - {}", message);
}
}
I'm trying to understand how many connections/socket connections are open in this case. I assume that these connections are long lived connection, I want to see if it is possible to can reuse these connection with some kind of connection pool. Thanks in advance
Solution 1:[1]
The clients first connect to one of the bootstrap servers; the server responds with the advertised listeners for each broker and the clients then connect to those servers that are the leaders for individual topics/partitions; AFAIK, there is no way to share these connections.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|---|
| Solution 1 | Gary Russell |
