'MongoDB ChangeStream vs. Apache Kafka

I want to be able to continue processing data from MongoDB after ensuring the upsert writes to it (by me) were successful. I have two options to accomplish this:

  • Write to Kafka after the writes to MongoDB were successful (from the same job that wrote to Mongo)
  • Receive the events of the written documents through Mongo ChangeStream, and continue processing them from there

As for my understanding, I see that the advantages for Kafka are that it is distributed and enables to read from more than one instance (I understood ChangeStream doesn't easily let this one). The ChangeStream advantage I see is that it lets me know what kind of operation was that (I perform upsert, so that lets me get to know if each upsert was an insert or an update). I'm not asking which is better because it clearly serves different use cases. But are there any more features or disadvantages for any of these options that I'm missing here in my consideration?

I also understand that both enables to continue after the client's reads failed for a while (Kafka within its quota and ChangeStream with the resume token)



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source