'Accessing Bitnami Kafka in K8s Cluster from Host?

After running:

helm repo add bitnami https://charts.bitnami.com/bitnami
helm install my-release bitnami/kafka

I then got pods:

$kubectl get pods
NAME                     READY   STATUS    RESTARTS   AGE
my-release-kafka-0       1/1     Running   2          24m
my-release-zookeeper-0   1/1     Running   0          24m

I attempted to access the Kafka running in my cluster to list topics:

$kafka-topics --bootstrap-server localhost:9092 --list
[2022-05-15 22:31:05,190] WARN [AdminClient clientId=adminclient-1] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)

It failed. I then tried to forward port 9092 from the guest to the host:

$kubectl port-forward my-release-kafka-0 9092:9092
Forwarding from 127.0.0.1:9092 -> 9092
Forwarding from [::1]:9092 -> 9092

In another tab, I tried and failed again:

$kafka-topics --bootstrap-server localhost:9092 --list
[2022-05-15 22:31:35,288] WARN [AdminClient clientId=adminclient-1] Error connecting to node my-release-kafka-0.my-release-kafka-headless.default.svc.cluster.local:9092 (id: 0 rack: null) (org.apache.kafka.clients.NetworkClient)
java.net.UnknownHostException: my-release-kafka-0.my-release-kafka-headless.default.svc.cluster.local: nodename nor servname provided, or not known
    at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method)
    at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:929)

What is the meaning of:

Error connecting to node my-release-kafka-0.my-release-kafka-headless.default.svc.cluster.local:9092

and how can I address this problem, i.e. so that I can access my cluster's Kafka from my local, the host?

kubernetesapache-kafkakubernetes-helm


Solution 1:[1]

It returns the pod DNS name because that's the advertised.listeners setting on the broker.

I'd recommend using Strimzi Operator rather than Bitnami Helm Charts, and reading the documentation on it about accessing the brokers using LoadBalancer or Ingress resources

Or decipher what you need from the Traffic Exposure parameters - https://github.com/bitnami/charts/tree/master/bitnami/kafka/#traffic-exposure-parameters + https://github.com/bitnami/charts/tree/master/bitnami/kafka/#accessing-kafka-brokers-from-outside-the-cluster

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1

Related Questions