'cannot make concurrent http requests using Camel kafka consumer with Threads EIP

in our application we use camel to consume message from kafka and in the end send http request to a server.

we tried to use the Threads EIP to send the http requests concurrently. but we found out that the http requests are executed one at a time and only when we get a reponse from the server the the next request will be sent.

simplified route definition looks like this:

from("kafka:topic&consumersCount=5&....")
                .threads(10, 10,"http-producer").maxQueueSize(200)
                .setHeader(Exchange.HTTP_METHOD, constant("GET"))
                .to("http://xxxx");

what we also found out:

  1. the issue seems to be the combination of kafka component and Threads EIP. we tried to use a timer component to generate the message instead of reading them from kafka. In that case the http requests are made concurrently

  2. we found some work-arround to this issue but still would like to understand why the original route doesn't work:

    a. if we add a aggregator before the http component the http requests are also made concurrently

    b. using the seda component the http requests are also made concurrently



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source