How to parse a json payload while sending it from producer to consumer topic?

Question

I want to send a payload from a producer topic to a consumer topic. I've created the channels locally & tried sending payload on producer topic. But the payload is not received on the consumer side.

I think this could be another in JSON formatting I've tried online JSON beautifiers but this is not helping.

Although it's a very slight chance, there is a possibility that there's something wrong with the code and the producer topic is not able to receive the payload. But I'm not able to confirm this.

Answer 1

You'll need to show code to solve your specific problem, but here is a simple example using kcat and jq

Producing

$ kcat -P -b localhost:9092 -t example
{"hello":"world"}
{"hello":"test data"}

Consume and parse

$ kcat -b localhost:9092 -C -t example -u | jq -r .hello
world
test data

The Kafka broker will not validate your JSON. The serialization library in your client might. So your issue could be any one of the following

• If your serializer failed, and you aren't catching and logging that exception
• You are not sending enough data for the producer buffer to clear, and so you should call .flush() method on the producer at some point.
• You have some Kafka authorization enabled on your cluster and your producer is failing to connect/produce.
• Some other connection setting is wrong in your code.