My Structured Spark Streaming program is to read JSON data from Kafka and write to HDFS in JSON format. I am able to save JSON to HDFS but it saves the JSON st
I use this kafka configuration with spring cloud and spring boot 2.6.6: @Configuration @RefreshScope public class KafkaProducerConfig { @Bean(name = "nativeP
I've tried to get the offset from Kafka topic based on timestamp when I tried to run it was throwing null pointer error, Map<TopicPartition, Long> timest
I'm working in an scenario where duplicated messages could arrive at a consumer (a KStream application). To use the typical case let's suppose it's an OrderCrea
I have a Kafka cluster with a topic that receives thousands of messages a day and I want to see how many messages went in the topic per date. I'm using JMX expo
I'm a bit new in Java so I would appreciate advice to deal with multiple conditions in Kafka Predicates. I've the following code which I'm able to have dynamic
I have a topic that will eventually have lots of different schemas on it. For now it just has the one. I've created a connect job via REST like this: { "name"
I'm trying to read data from one kafka topic and writing to another after making some processing. I'm able to read data and process it when i try to write it to
From Kafka doc https://kafka.apache.org/11/documentation.html#producerconfigs , it says that: The configuration controls the maximum amount of time the client
I am using debezium oracle connector in kafka connect.While starting connector I am getting below error, java.lang.RuntimeException: Failed to resolve Oracle da
I am new to helm and I am trying to generate different topics for kafka with a range function to not have a yaml file for each topic: I have different topics (t
The image below is my topic. Every once in a while, a value other than an empty or json is returned. If it is null, it throws an error as below and enters the
I'm learning how to write tests and especially tests that have a producer in it. I cannot post all the classes because it's HUGE (and not mine, I should just pr
I have to add encryption and authentication with SSL in kafka. This is what I have done: Generate certificate for each broker kafka: keytool -keystore server
I have been try to run Zookeeper and Kafka on Docker container. I got a lot of errors [error occurred during error reporting , id 0xb] and [Too many errors, abo
See Update below to show potential workaround Our application consumes 2 topics as KTables, performs a left join, and outputs to a topic. During testing, we fou
I'm trying to build a connector to go into the confluent kafka library. I have seen many examples in java that use configDef to define the configuration options
I'm doing real time streaming on Twitter and wonder is there a way to extract only messages and certain values from Kafka topic?
I want to create I stream kafka consumer in pyFlink, which can read tweets data after deserialization (json), I have pyflink version 1.14.4 (last version) Can I
I have a defauldict I want to serialize as json. But some characters then replaced with unicode escaped charasters. For example, if I have string "International