Micronaut kafka project - mutiple consumers each with different bootstrap server and ssl certs

Question

I am trying to set up a micronaut project with multiple consumers each with different bootstrap server and ssl certs. I am not setting global bootstrap server and certs. This does not work. Any suggestion is appreciated.

One another option is to combine the certs into one jks file and set global bootstrap and ssl configuration.

kafka:
  consumers:
    group1:
      bootstrap:
        servers: $someserver1
      ssl:
        keystore:
          location: /keystore.jks
          password: password
        truststore:
          location: /truststore.jks
          password: password
          type: PKCS12
      security:
        protocol: ssl
    group2:
      bootstrap:
        servers: $someserver2
      ssl:
        keystore:
          location: /keystore-1.jks
          password: password
        truststore:
          location: /truststore-1.jks
          password: password
          type: PKCS12
      security:
        protocol: ssl

Answer 1

The issue is that consumer classes are not annotated quite correctly. Here is what worked for me:

In application.yml (note that both servers use the same SSL certs):

kafka:
  consumers:
    group1:
      topic: some-topic-1
      bootstrap:
        servers: server-1:9092
    group2:
      topic: some-topic-2
      bootstrap:
        servers: server-2:9092
  ssl:
    keystore:
      location: /keystore.jks
      password: "password1"
    truststore:
      location: /truststore.jks
      password: "password2"
  security:
    protocol: ssl

Then, annotate your classes to set consumer specific properties. Example in Kotlin:

@KafkaListener(
    properties = [Property(name = ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, value = "\${kafka.consumers.group1.bootstrap.servers}")],
)
class TestConsumer1() {
    @Topic("\${kafka.consumers.group1.topic}")
    fun receiveMessage(record: ConsumerRecord<String, String>, acknowledgement: Acknowledgement) {
        TODO("Handle events from server-1:9092 and some-topic-1 :)")
    }
}

More information on the @KafkaListener and properties can be found here: https://micronaut-projects.github.io/micronaut-kafka/latest/guide/#kafkaListenerConfiguration