'What is the best approach to scale nodejs that consumes kafka messages

I have a node application that consumes Kafka messages, makes some long operations, and finally, sends the result to another microservice through Kafka message.

what is the best approach to improve my application scalability so it can process more Kafka messages per second? and why?

  1. Using the clustering module of node js to create multiple instances that connect to Kafka using the same consumer group. (I think it should enhance the scalability by cores count)

  2. Using a thread pool of the worker threads node js module. and distribute the Kafka consumed messages across the worker threads.

  3. any other suggestions?

node.jsmultithreadingapache-kafka


Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source

Related Questions