I am trying to consume messages from Kafka and publish them to Google PubSub. We have 4 Kafka concurrent consumer threads and I injected the Google pubsub clien
I am trying to consume messages from Kafka and publish them to Google PubSub. We have 4 Kafka concurrent consumer threads and I injected the Google pubsub clien
I need to create an application which has to bridge multiple Queue solutions like Rabbitmq, Kafka, Azure eventhub. I am thinking to use Spring cloud Stream so t
I use istio-ingress gateway and virtualservice to expose different microservices. So far all of them have been http services, so it was straight-forward to foll
I am using version 2.13. Kafka is able to produce messages but not able to consume messages after investigation I found that consumer groups are not getting cre
I am currently using Ksqldb in the k8s environment. I am using TABLE for aggregate operations. Since it is a stateful operation, ksqldb is expected to be using
I have a Kafka cluster and a spring boot application that is configured for EOS. The application consumes from topic A performs some business logic then produce
I am using flink with v1.13.2 . And I am trying to migrate FlinkKafkaConsumer to KafkaSource. While i am testing new KafkaSource, i am getting the following exc
I have an ETL (spark-scala). After writing in a table, a message with "header" must be sent to Kafka. I couldn't add the header in the message. I have a spark D
I am newbie in kafka, i try build a service send mail with attach files. Execution flow: Kafka will receive a message to send mail function get file will downlo
A friend reached out and told me that he'd built a data pipeline that captures changes from MySQL using the Debezium connector, and then performs streaming aggr
I want to get state store with custom key and value. I have a kafka topic example-kafka-topic-event. This is how I get ktable at the code level: @Component("exa
I am setting up Strimzi kafka Mirrormaker2 in our test environment which receives on an average 100k messages/5 mins. we have around 25 topics and 900 partition
docker pull confluentinc/cp-kafka I use this code to pull the kafka image. Then, use the following to start the containers. docker run --rm -d --name kafka-se
I have a relatively simple setup where I'm running a dynamic number of instances of my stateless Kafka streams app based on my consumer group lag. This is achie
Versions Sarama - v1.32.0 Kafka - 5.4.6-2.12 Go - v1.16.2 sarama.Logger = log.New(os.Stderr, "[Sarama] ", log.LstdFlags) brokers := []string{ "kafka1
Kafka 0.11.0 Documentation mentions the normal values for the MBean "RequestMetrics.RemoteTimeMs" as "non-zero for produce requests when ack=-1" Link to officia
Spring Kafka Batch consumer receives only one or two messages we have increased fetch.min.bytes to 9000 and fetch.max.wait.ms 5000 [Based on this answ
I'm using python queue to insert data packets from mqtt listeners But I'm not sure when this queue will be loaded by Mqtt packet. Can we put a listener on to th
In docker image johnnypark/kafka-zookeeper new topics are not created automatically. How do I set auto.create.topics.enable=true in this docker image johnnypark