'KAFKA Connect XML source and JSON sink

Is there a way in KAFKA to consume XML source and convert it to JSON and send JSON data to KAFKA to sink?

I have seen Avro, Protobuf as convertors in kafka connect? Are they capable of converting XML to JSON? or would they convert to AVRO, Protobuf specific formats rather than JSON?

jsonxmlapache-kafkaapache-kafka-connect


Solution 1:[1]

Kafka Connect can use any data format. However, there is no builtin Converter for XML, so you just use StringConverter.

Then you can use transforms to parse the XML into an internal format Connect works with, known as Struct. E.g. https://jcustenborder.github.io/kafka-connect-documentation/projects/kafka-connect-transform-xml/transformations/examples/FromXml.books.html

(the json "input" and "output" shown is from a REST proxy request, only look at the value field)

When written to a Connect sink, you can then use JSONConverter (or any other one) to deserialize the internal Struct object

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1

Related Questions