I'm using open-telemetry to trace my applications and have a few microservices and Kafka broker in my distributed system. I'm using Java/spring-boot and in the
load multiple postgresql tables into multiple kafka topics in google cloud environment using pubsub or kafka connect.
I am trying to read a stream from kafka using pyspark. I am using spark version 3.0.0-preview2 and spark-streaming-kafka-0-10_2.12 Before this I just stat zoo
I have a Stream on a topic with schema: --root --name: string --age: integer --accounts: Array --email I would like to select all root elements hav
I have a Stream on a topic with schema: --root --name: string --age: integer --accounts: Array --email I would like to select all root elements hav
Sometimes we encounter lag in kafka consumer due to some external issues. Flink job will always consume kafka history (delayed data) with exactly-once semantics
Sometimes we encounter lag in kafka consumer due to some external issues. Flink job will always consume kafka history (delayed data) with exactly-once semantics
I use Spring for Apache Kafka. I'd like to stop listening to my topic and wait to escape OOM. How can I do it?
Code I have that successfully connects to Kafka with an SSL connection in Python 3.6.7 fails when using Python 3.7.3, with error message SSL: WRONG_VERSION_NUMB
We use 3 node kafka clusters running 2.7.0 with quite high number of topics and partitions. Almost all the topics have only 1 partition and replication factor o
I have setup Apache MirrorMaker 3.0.0 with active-active strategy for two Kafka clusters (named DC, DR). So topic on DC is replicated by MirrorMaker2 as DC.<
I am trying to do link between kafka and spark by reading data from one topic and tryy to print the content of this topic into a DataFrame, but by doing connect
I have already gone though Robin Moffat's blog, and several SO posts on the same subject, but still my configuration doesnt work. My docker-compose.yml: kafka
How to start Zookeeper and Kafka broker on remote target with Ansible Playbook. Following commands work fine locally. Start Zookeeper: cd /opt/kafka ./bin/zooke
My Structured Spark Streaming program is to read JSON data from Kafka and write to HDFS in JSON format. I am able to save JSON to HDFS but it saves the JSON st
I use this kafka configuration with spring cloud and spring boot 2.6.6: @Configuration @RefreshScope public class KafkaProducerConfig { @Bean(name = "nativeP
I've tried to get the offset from Kafka topic based on timestamp when I tried to run it was throwing null pointer error, Map<TopicPartition, Long> timest
I'm working in an scenario where duplicated messages could arrive at a consumer (a KStream application). To use the typical case let's suppose it's an OrderCrea
I have a Kafka cluster with a topic that receives thousands of messages a day and I want to see how many messages went in the topic per date. I'm using JMX expo
I'm a bit new in Java so I would appreciate advice to deal with multiple conditions in Kafka Predicates. I've the following code which I'm able to have dynamic