I am newbie in kafka, i try build a service send mail with attach files. Execution flow: Kafka will receive a message to send mail function get file will downlo
A friend reached out and told me that he'd built a data pipeline that captures changes from MySQL using the Debezium connector, and then performs streaming aggr
I want to get state store with custom key and value. I have a kafka topic example-kafka-topic-event. This is how I get ktable at the code level: @Component("exa
I am setting up Strimzi kafka Mirrormaker2 in our test environment which receives on an average 100k messages/5 mins. we have around 25 topics and 900 partition
docker pull confluentinc/cp-kafka I use this code to pull the kafka image. Then, use the following to start the containers. docker run --rm -d --name kafka-se
I have a relatively simple setup where I'm running a dynamic number of instances of my stateless Kafka streams app based on my consumer group lag. This is achie
Versions Sarama - v1.32.0 Kafka - 5.4.6-2.12 Go - v1.16.2 sarama.Logger = log.New(os.Stderr, "[Sarama] ", log.LstdFlags) brokers := []string{ "kafka1
Kafka 0.11.0 Documentation mentions the normal values for the MBean "RequestMetrics.RemoteTimeMs" as "non-zero for produce requests when ack=-1" Link to officia
Spring Kafka Batch consumer receives only one or two messages we have increased fetch.min.bytes to 9000 and fetch.max.wait.ms 5000 [Based on this answ
I'm using python queue to insert data packets from mqtt listeners But I'm not sure when this queue will be loaded by Mqtt packet. Can we put a listener on to th
In docker image johnnypark/kafka-zookeeper new topics are not created automatically. How do I set auto.create.topics.enable=true in this docker image johnnypark
I use macOS M1 Big Sur 11.2.3, but my kafka cannot running well and cannot create/list the topics. I don't know its because the OS or not, but the log for kafka
Hi I am trying to run this code in but it is working fine in another EC2 Azkaban instance but not giving below error for another instance. private val adminprop
Setup I've installed latest (7.0.1) version of Confluent platform in standalone mode on Ubuntu virtual machine. Python producer for Avro format Using this sampl
On setting kafka producer property - enable.idempotence to true kafkaProps.put("enable.idempotence" , "true"); I am getting below error - 2021-04-18 16:43:53.58
My Kafka Consumer says the following: [TopicPartition{topic=my-topic,partition=0,offset=-1000,error=None}] Whenever I run poll(), it returns None. I want the o
I have files that will be coming in daily that I would like to process as they come in and insert into existing sql tables (using postgres). What is the best wa
I am new to Apache flume https://flume.apache.org/. For one of the use-case, I need to move data from the Kafka topic on one cluster (bootstrap: bootstrap1, top
I want to use the PLC4X Connector (https://www.confluent.io/hub/apache/kafka-connect-plc4x-plc4j) to connect OPC UA (Prosys Simulation Server) with Kafka. Howev
I am setting up Kafka on my local Windows 10 machine. So downloaded all the required binaries and updated the two settings server and zookeeper properties as pe