'Architecture advise for newsfeed system?

Our team is currently developing a system similar to a news feed or notification system

We're stuck on the implementation. Our system should work as follows

User logs into the application, connects to the service using SSE connection service sends him all the unreceived data when he was offline, as well as sends all new records received at the moment

Current architecture is presented on the image enter image description here

Implementation description:

  1. User opens the application and asks for news
  2. Service A creates a consumer for the user with a unique group id and subscribes to the news topic
  3. In accordance with the previous offset, we get the news that the user has not already received and continue to listen to the topic while the SSE connection is active

Some remarks:

  • Service B in this case is a certain news service which writes news to Kafka
  • We should not lose news that was sent by Service B while the user was offline
  • In our project we use the Java + Spring WebFlux
  • One of the disadvantages of this implementation that colleagues have told me about is the large overhead to create and maintain a separate consumer for each client

The questions are as follows:

  1. What are the disadvantages of our solution?
  2. What are the alternatives for implementing a news feed?
apache-kafkaarchitecturebackend


Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source

Related Questions