Category "apache-kafka"

what is difference between partition and replica of a topic in kafka cluster

What is difference between partition and replica of a topic in kafka cluster. I mean both store the copies of messages in a topic. Then what is the real diffre

Better Event centric way to solve request-reply problem in Kafka or any streaming service

I am stuck in a problem while using Kafka in a microservice architecture . I am not able to understand how a microservice handling HTTP requests will be able to

Is this the right Kafka Consumer Config - under this Kafka setup?

My Kafka Producer is producing messages at the rate of about .. 350 mb per 30 seconds.. Kafka Setup: --> 1 Zookeeper instance --> 3 Kafka Brokers --&

Spark Streaming "ERROR JobScheduler: error in job generator"

I build a spark Streaming application to keep receiving messages from Kafka and then write them into a table HBase. This app runs pretty good for first 25 mins

Kafka MirrorMaker - from Older version to Newer Version

I want to mirroring from Kafka Source Cluster to Kafka Destination Cluster. Everything is working fine if my both Source and Target Cluster are on the same vers

How to stream data from Kafka to MongoDB by Kafka Connector

I want to stream data from Kafka to MongoDB by using Kafka Connector. I found this one https://github.com/hpgrahsl/kafka-connect-mongodb. But there is no step t

Spark streaming jdbc read the stream as and when data comes - Data source jdbc does not support streamed reading

I am using PostGre as database. I want to capture one table data for each batch and convert it as parquet file and store in to s3. I tried to connect using JDB

Unable to read additional data from client session id

We have hadoop cluster with 3 kafka machines and 3 zookeeper servers hadoop version - 2.6.4 ( HORTONWORKS ) under zookeeper logs ( /var/log/zookper ) we saw a m

Kafka vs StreamSets

I was reading articles related to Kafka and StreamSets and my understanding was Kafka acts as a broker between Producer system and subscriber. Producer push t

Kafka vs StreamSets

I was reading articles related to Kafka and StreamSets and my understanding was Kafka acts as a broker between Producer system and subscriber. Producer push t

Kafka vs StreamSets

I was reading articles related to Kafka and StreamSets and my understanding was Kafka acts as a broker between Producer system and subscriber. Producer push t