Category "apache-kafka"

Are kafka acks received in the same order of produced messages

I'm working on a process that collect data from IBM MQ and process it to a kafka topic. To make sure not loosing any message,I need to commit my JMS message onl

When collecting data with Modbus protocol through kafka Producer, collection stops after a certain period of time

I have deployed a Kafka cluster on a GCP instance. I used the connector through config/connect-distributed.properties. Start collecting data through restapi usi

Bean instantiation via factory method failed; exception org.springframework.beans.BeanInstantiationException

I have the following Spring configuration for Kafka: import org.apache.kafka.clients.producer.ProducerConfig; import org.springframework.boot.context.properties

Is kafka log cleaner logging anything on AWS MSK?

I use AWS MSK cluster with brokers logging turned on to CloudWatch. Logging works and I can see brokers logs. We have some topics with cleanup.policy=compact an

How can a org.apache.kafka.connect.data.Decimal stored in an avro file be converted to a python type?

I am trying to interpret a Avro record stored by Debezium in Kafka, using Python { "name": "id", "type": {

multi-broker kafka InvalidReplicationFactorException

I'm having some problems while getting kafka to work in a multi-broker cluster. It's managed by Ambari 2.6.3.0 and I installed kafka broker in both hosts and I

Unable to deserialize Kafka stream to pojo. Could not find class specified in writer's schema

Getting exception while reading from kafka topic: Caused by: org.apache.kafka.common.errors.SerializationException: Error deserializing Avro message for

Kafka server not returning anything

Zookeper is up and running in windows env.Now when i start Kafka server using .\bin\windows\kafka-server-start.bat .\config\server.properties nothing is dis

How to send messages synchronously in kafka?

One way of achieving it could be by setting the properties parametermax.in.flight.requests.per.connection = 1. But I want to know if there is an even direct or

Consume events from AWS EventBridge in a self hosted kafka cluster outsite aws

We got a SaaS which is publishing it's events on AWS eventbridge (coulple of milion per day). We would love to consume those events and put them on our self hos

Kafka Consumer to read from multiple topics

I am very new to Kafka. I am creating two topics and publishing on these two topics from two Producers. I have one consumer which consumes the messages from bot

Apache Kafka cannot start multiple instances on same local machine

I am trying to set up Apache Kafka on my local machine to try it out following this official guide: https://kafka.apache.org/quickstart. However, when I tried

How to write a KafkaAvro Serde for GenericData.Record

I'm using Kafka 0.10.2 and Avro for the serialization of my messages, both for the key and for the value data. Now I would like to use Kafka Streams but I'm stu

Confluent Maven repository not working?

I need to use the Confluent kafka-avro-serializer Maven artifact. From the official guide I should add this repository to my Maven pom <repository> <

kafka deployment on strimzi

i'm trying to deploy kafka with strimzi, problem is, its exposing kafka brokers as load balancers and assigning them an external IP. i want kafka brokers to be

only one client consumed all message in kafka consumer group

There is 3 (node.js + kafka-node) client Following the tutorial, I made topic 'quickstart-events'. Topic: quickstart-events TopicId: oZC9g7FvTiOm-QIuPpyo

Why retention.ms of Kaka Streams repartition topic is set to -1 by default? Isn't this infinitely retain messages in repartition topic?

I think it's related to the below links, but I don't understand. https://issues.apache.org/jira/browse/KAFKA-6535 https://issues.apache.org/jira/browse/KAFKA-61

Kafka producer Interceptor

I am trying add a Interceptor to do validation for the messages published by the producer to Kafka topic. I need to do few validations in addition to Schema val

Kafka Error while creating ACLs NoAuth for /kafka-acl/TransactionalId

I am trying to setup my very first Kafka cluster using confluent 6.0.1 community edition. I have three zookeeper and three kafka nodes. Three server nodes are:

Kafka cluster security for IOT

I am new to the Kafka and want to deploy Kafka Production cluster for IOT. We will be receiving messages from Raspberry Pi over the internet to our Kafka cluste