'Kafka SASL_SSL Authentication error, how authenticate?

I'm working in Java with a pipeline that consumes data from a Kafka Cluster, it's available on

https://github.com/confluentinc/confluent-google-examples/blob/master/ccloud-dataflow-demo/entry-df-pipeline/src/main/java/com/ecuevas/DataflowPipeline.java

According to the repository, the props and pipeline are defined:

        Map<String, Object> props = new HashMap<>();
        props.put("auto.offset.reset", "earliest");
        props.put("ssl.endpoint.identification.algorithm", "https");
        props.put("sasl.mechanism", "PLAIN");
        props.put("request.timeout.ms", 20000);
        props.put("retry.backoff.ms", 500);
        props.put("security.protocol", "SASL_SSL");
        props.put("sasl.jaas.config",String.format("org.apache.kafka.common.security.
        plain.PlainLoginModule required username=\"%s\" password=\"%s\";",username, password));

        LogKafkaMsg logKafkaMsg = new LogKafkaMsg();

        Pipeline pipeline = Pipeline.create(options);

        PCollection<KV<String, String>> entries =
                pipeline
                        .apply(
                                "Read Entries from Confluent Cloud Topic",
                                KafkaIO.<String, String>read()
                                        .withBootstrapServers("<your-bootstrap-server>")
                                        .withTopic("entries")
                                        .withConsumerConfigUpdates(props)
                                        .withKeyDeserializer(StringDeserializer.class)
                                        .withValueDeserializer(StringDeserializer.class)
                                        .withoutMetadata()
                        );

My problem is when the code is executed, returns the following error:

at org.apache.beam.runners.dataflow.DataflowRunner.run(DataflowRunner.java:969)
        at org.apache.beam.runners.dataflow.DataflowRunner.run(DataflowRunner.java:198)
        at org.apache.beam.sdk.Pipeline.run(Pipeline.java:322)
        at org.apache.beam.sdk.Pipeline.run(Pipeline.java:308)
        at com.ecuevas.DataflowPipeline.main(DataflowPipeline.java:214)
Caused by: org.apache.kafka.common.errors.SaslAuthenticationException: Invalid SASL/PLAIN response: expected 3 tokens, got 4

Probably the issue is related with the SASL/PLAIN authentication, but I don't know a possible solution. I think that is related with:

  • SSL authentication: Include ca-cert, ca-key, ca-password or another special.
  • Dependencies updates: The libraries versions aren't updated.
javaauthenticationsslapache-kafkaapache-beam


Solution 1:[1]

Seems like Dataflow VMs are not authenticated to connect to the Kafka cluster. Have you tried authenticating using a keystore certificate file per instructions here ?

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1 chamikara

Related Questions