Data streaming in Apache-Ignite through Apache-Kafka consumes high CPU

Question

Data streaming in Apache-Ignite through Apache-Kafka consumes high CPU

I am using Ignite Source Connector (Single Kafka connector node) to export the Ignite Events from my ignite cluster(2 Nodes) to Kafka broker nodes (2 Nodes) with single topic and 29 partitions.

I have processed 0.1 Million events (PUT, DELETE) per minute with the 1k size messages(in Avg).

My connector node consumes 90% of the CPU (you can see it below).

My Connector node machine config.

RAM --> 30 GB
ROM --> 1TB
Configured Heap --> 15GB
No of CPU Core --> 40 (20 x 2)



#connector
name=my-ignite-source-connector
connector.class=class of my source connector
tasks.max=1
topicNames=ignite-data

#cache
cacheName=DOCIDS
cacheAllowOverwrite=true
cacheEvts=put , removed
evtBatchSize=100
numberOfPartitions=29
igniteCfg=/myconfig/ignite-config.xml

                 

<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:util="http://www.springframework.org/schema/util"
       xsi:schemaLocation="
        http://www.springframework.org/schema/beans
        http://www.springframework.org/schema/beans/spring-beans.xsd
        http://www.springframework.org/schema/util
        http://www.springframework.org/schema/util/spring-util.xsd">
    <bean abstract="true" id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">
        <!-- Set to true to enable distributed class loading for examples, default is false. -->
        <property name="communicationSpi"><bean class="org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi"><property name="socketWriteTimeout" value="60000"/></bean></property><property name="peerClassLoadingEnabled" value="true"/><property name="clientFailureDetectionTimeout" value="10000"/><property name="dataStorageConfiguration"><bean class="org.apache.ignite.configuration.DataStorageConfiguration"><property name="walPath" value="wal_data/"/><property name="walArchivePath" value="wal_data/"/><property name="defaultDataRegionConfiguration"><bean class="org.apache.ignite.configuration.DataRegionConfiguration"><property name="persistenceEnabled" value="true"/><property name="maxSize" value="#{15L * 1024 * 1024 * 1024}"/></bean></property></bean></property>

        <!-- Enable task execution events for examples. -->
        <property name="includeEventTypes">
            <list>
                <!--Task execution events-->
                <util:constant static-field="org.apache.ignite.events.EventType.EVT_TASK_STARTED"/>
                <util:constant static-field="org.apache.ignite.events.EventType.EVT_TASK_FINISHED"/>
                <util:constant static-field="org.apache.ignite.events.EventType.EVT_TASK_FAILED"/>
                <util:constant static-field="org.apache.ignite.events.EventType.EVT_CACHE_OBJECT_PUT"/>
                <util:constant static-field="org.apache.ignite.events.EventType.EVT_CACHE_OBJECT_REMOVED"/>
                <!--Cache events-->



            </list>
        </property>
 <!-- Explicitly configure TCP discovery SPI to provide list of initial nodes. -->
        <property name="discoverySpi">
            <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
                <property name="ipFinder">
                    <!--
                        Ignite provides several options for automatic discovery that can be used
                        instead os static IP based discovery. For information on all options refer
                        to our documentation: http://apacheignite.readme.io/docs/cluster-config
                    -->
                    <!-- Uncomment static IP finder to enable static-based discovery of initial nodes. -->
                    <!--<bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">-->
                    <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
                        <property name="addresses">
                            <list>
                                <!-- In distributed environment, replace with actual host IP address. -->
                                <value>ignite-server-node1:47500..47509</value><value>ignite-server-node2:47500..47509</value><value>kafka-conect-node:47500..47509</value>
                            </list>
                        </property>
                    </bean>
                </property>
            </bean>
        </property>
    </bean>
</beans>

The data transfer process is working as intended only for a few hours because of this high CPU consumption, after that the Kafka connector node will be OFFLINE in ignite server topology.

Any pointers will help.

Thanks in advance.