Category "apache-kafka"

Connect to confluent schema registry SSL configuration - unable to find valid certification path to requested target

How should it be configured the spring-cloud-schema-registry-client to communicate with Confluent Schema Registry on a secured manner using SSL truststore? My c

Half of messages lost when Spark Streaming (4 concurrent drivers) reads from Kafka and writes to MongoDB

I have set up a docker network with: 4 producer containers (each scraping a different forum) that produce to a Kafka container (1 topic for each of the forum) A

Too many org.apache.kafka.common.metrics.stats.* instance been loaded caused OOM issue

we have project using kafka and kafka stream, recently we found OOM issue happen for the project, after analysis the dump file, we found that too many 'org.apac

Embedded Kafka contract tests randomly failing

I am using the spring cloud contrat to test a kafkaListsener of my service. I use EmbeddedKafka together with the spring cloud contract. The listener in my serv

Kafka deployed through istio cannot be provided to the k8s cluster for external access

Kafka deployed through istio cannot be provided to the k8s cluster for external access with the following error Error connecting to node kafka-0.kafka-headless

Kafka org.apache.kafka.common.errors.TimeoutException for some of the records

Getting below error message when producing record to Azure event hub(kafka enabled) Expiring 14 record(s) for eventhubname: 30125 ms has passed since batch cre

No data in the changelog topic for a KSQL table

Step 1. Postgres CDC Data I have a Postgres table which has been set up to use Debezium PostgreSQL Connector to generate CDC. In Kafka, I can see the CDC data i

Read from single kafka topic and upsert into multiple oracle tables

I'm using kafka to send the multiple collections data into a single topic using mongo source connector and upsert the data into different oracle tables using jb

kafka and parallel consumer: why order is important into a microservice architecture

I started to dive into kafka ecosystem. I was surprised to find out that by default, each consumer only digests one "event" at a time, sequentially! It's given

Kafka Topic - filter and dispatch messages

Background Our software solution collects data ("events") per customer. Some of the customers (a small fraction ~3%) ask to get this data into "their systems" (

Spring Batch - Resume already started job when master fails

I didn't find an answer to my question so far so I'm giving it a try here. Let's assume a Spring Batch application with remote partitioning. There's one master/

Spring Apache Kafka onFailure Callback of KafkaTemplate not fired on connection error

I'm experimenting a lot with Apache Kafka in a Spring Boot App at the moment. My current goal is to write a REST endpoint that takes in some message payload, wh

Running Kafka Confluent Platform on WSL 2 (Ubuntu Distribution) and Spring application on Windows (Broker may not be available)

I'm running Confluent Platform in wsl 2(Ubuntu Distribution) and I also running a Spring application on Windows but when I send a message with a producer I have

Best way to update/delete records in db based on Apache Kafka topic message

I have an application publishing messages on a Kafka Topic. I've used Confluent's Snow Sync to synchronize the topic with a Snowflake table so when I publish a

Can we create ksql table from topic together with topic data?

I have a db connector, which will detect the timestamp column update on my user table on my database and push it to my kafka topic. Thus, my kafka topic will ke

How to use Kafka with SSL via logback appander?

I use this logback appender to send logs to Kafka: https://github.com/danielwegener/logback-kafka-appender When Kafka was PLAINTEXT everything worked correctly.

Topic authorization failed error while KSQL tries connecting to cluster

I am trying to have my ksqldb-server docker instance up and connect to a remote Kafka cluster, but getting an error. Here are the details docker-compose.yml ---

confluent 5.0.1 upgrade to 7.0.1

Trying to upgrade confluent I have checked upgrade link: https://docs.confluent.io/platform/7.0.1/installation/upgrade.html I found inside it said in the Prepar

Not able to open Control Center UI on localhost

I am trying to understand confluent kafka and do some hands on with it Trying to setup on my local machine . I have followed the instructions in this doc : http

KSQL Rest INSERT not inserting

I have created a KSQL table with the following command: ksql> CREATE TABLE CUSTTABLE (id INT PRIMARY KEY, name VARCHAR, purchase VARCHAR) WITH (KAFKA_TOPIC =