'Strimzi Kafka connector gets SocketTimeoutException while connecting to confluent schema registry
I created a Kafka connect cluster with Strimzi operator:
apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaConnect
metadata:
annotations:
strimzi.io/use-connector-resources: "true"
name: test-cluster-0
spec:
bootstrapServers: {{.Values.bootstrapServers}}
config:
group.id: connect-cluster-datagen
key.converter: io.confluent.connect.avro.AvroConverter
key.converter.schemas.enable: true
value.converter: io.confluent.connect.avro.AvroConverter
value.converter.schema.registry.url: http://cp-schema-registry.my-namespace.svc:8081
key.converter.schema.registry.url: http://cp-schema-registry.my-namespace.svc:8081
value.converter.schemas.enable: true
offset.storage.replication.factor: 3
offset.storage.topic: datagen-connect-cluster-offsets
status.storage.replication.factor: 3
status.storage.topic: datagen-connect-cluster-status
config.storage.replication.factor: 3
config.storage.topic: datagen-connect-cluster-configs
replicas: 1
version: 3.0.0
Schema registry is created using confluent helm charts -- with following values.yaml:
cp-kafka:
enabled: false
cp-zookeeper:
enabled: false
cp-kafka-rest:
enabled: false
cp-kafka-connect:
enabled: false
cp-control-center:
enabled: false
cp-ksql-server:
enabled: false
cp-schema-registry:
enabled: true
image: confluentinc/cp-schema-registry
kafka:
bootstrapServers: {{.Value.bootstrapServers}}
However connectors throw exceptions like:
2022-02-15 18:07:01,939 ERROR [datagen-source-topic-connector|task-0] Failed to send HTTP request to endpoint: http://cp-schema-registry.my-namespace.svc:8081/subjects/datagen-source-topic-value/versions (io.confluent.kafka.schemaregistry.client.rest.RestService) [task-thread-datagen-source-topic-connector-0]
java.net.SocketTimeoutException: connect timed out
at java.base/java.net.PlainSocketImpl.socketConnect(Native Method)
at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:399)
at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:242)
at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:224)
at java.base/java.net.Socket.connect(Socket.java:609)
This question mentions that SCHEMA_REGISTRY_HOST_NAME should be resolvable. However confluent helm chart sets this to pod ip -- which should be resolvable:
env:
- name: SCHEMA_REGISTRY_HOST_NAME
valueFrom:
fieldRef:
fieldPath: status.podIP
Am I missing a configuration parameter here? If I send normal curl requests, I see the following in schema-registry logs:
io.confluent.rest.exceptions.RestNotFoundException: Subject 'datagen-source-topic-value' not found.
at io.confluent.kafka.schemaregistry.rest.exceptions.Errors.subjectNotFoundException(Errors.java:69)
at io.confluent.kafka.schemaregistry.rest.resources.SubjectVersionsResource.listVersions(SubjectVersionsResource.java:197)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52)
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124)
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167)
at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:219)
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79)
at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:469)
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:391)
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:80)
at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:253)
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248)
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244)
at org.glassfish.jersey.internal.Errors.process(Errors.java:292)
at org.glassfish.jersey.internal.Errors.process(Errors.java:274)
at org.glassfish.jersey.internal.Errors.process(Errors.java:244)
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265)
at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:232)
at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680)
at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394)
at org.glassfish.jersey.servlet.ServletContainer.serviceImpl(ServletContainer.java:386)
at org.glassfish.jersey.servlet.ServletContainer.doFilter(ServletContainer.java:561)
at org.glassfish.jersey.servlet.ServletContainer.doFilter(ServletContainer.java:502)
at org.glassfish.jersey.servlet.ServletContainer.doFilter(ServletContainer.java:439)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
