'WildFly 17.0.1.Final HeapMemory Issues

I need some suggestions or guidelines on Wildfly version which is in production recently.

We see heap memory spikes very often and garbage collection is not working and heap gets filled in and reaches to max 2 GB (this is our allocation of heap) and we need to restart the application to be back on normal. When we analysed the memory dump we couldn't find any issues on application code, but we have found one class is taking more memory. That class is related with to ActiveMQ & WildFly integration which application uses MDB and we are on EJB 3.0. And please more that we use Amazon Corretto with WildFly. Please any one have observed this issue regarding the below object from WildFly.

When we analysed the stack traces and objects in memory, the below object holds more memory and it is from WildFly class. Someone please share any thoughts or idea on this.

We are using wildfly-17.0.1.Final and Corretto-8.212.04.2 (build 1.8.0_212-b04). And below is out JVM configuration

<jvms>
    <jvm name="default">
        <heap size="2048m" max-size="2048m"/>
        <jvm-options>
            <option value="-server"/>
            <option value="-XX:MetaspaceSize=256m"/>
            <option value="-XX:MaxMetaspaceSize=256m"/>
        </jvm-options>
    </jvm>

Heap stack trace as in below

**`**Who reference 1.25gb (71.7%) of byte[]?`**

1,196,563K **(65.3%) of byte[]** Java Static org.wildfly.extension.messaging.activemq.broadcast.CommandDispatcherBroadcastEndpointFactory.BROADCAST_MANAGERS {java.util.concurrent.ConcurrentHashMap}.values org.wildfly.extension.messaging.activemq.broadcast.QueueBroadcastManager.broadcasts {java.util.concurrent.LinkedBlockingDeque}**

We are running our J2EE application on Wildfly Cluster environment with the profile "full-ha" and please find our ACTIVEMQ configration

 <subsystem xmlns="urn:jboss:domain:messaging-activemq:7.0">
            <server name="default">
                <cluster password="password"/>
                <statistics enabled="${wildfly.messaging-activemq.statistics-enabled:${wildfly.statistics-enabled:false}}"/>
                <security-setting name="#">
                    <role name="guest" send="true" consume="true" create-non-durable-queue="true" delete-non-durable-queue="true"/>
                </security-setting>
                <address-setting name="#" dead-letter-address="jms.queue.DLQ" expiry-address="jms.queue.ExpiryQueue" max-size-bytes="10485760" page-size-bytes="2097152" message-counter-history-day-limit="10" redistribution-delay="1000"/>
                <http-connector name="http-connector" socket-binding="http" endpoint="http-acceptor"/>
                <http-connector name="http-connector-throughput" socket-binding="http" endpoint="http-acceptor-throughput">
                    <param name="batch-delay" value="50"/>
                </http-connector>
                <in-vm-connector name="in-vm" server-id="0">
                    <param name="buffer-pooling" value="false"/>
                </in-vm-connector>
                <http-acceptor name="http-acceptor" http-listener="default"/>
                <http-acceptor name="http-acceptor-throughput" http-listener="default">
                    <param name="batch-delay" value="50"/>
                    <param name="direct-deliver" value="false"/>
                </http-acceptor>
                <in-vm-acceptor name="in-vm" server-id="0">
                    <param name="buffer-pooling" value="false"/>
                </in-vm-acceptor>
                <broadcast-group name="bg-group1" jgroups-cluster="activemq-cluster" connectors="http-connector"/>
                <discovery-group name="dg-group1" jgroups-cluster="activemq-cluster"/>
                <cluster-connection name="my-cluster" address="jms" connector-name="http-connector" discovery-group="bg-group1"/>
                <jms-queue name="ExpiryQueue" entries="java:/jms/queue/ExpiryQueue"/>
                <jms-queue name="DLQ" entries="java:/jms/queue/DLQ"/>
                <jms-queue name="CSVProcessPendingMessageBean" entries="java:/jms/queue/CSV_PROCESS_PENDING_Q java:jboss/exported/jms/queue/CSV_PROCESS_PENDING_Q"/>
                <jms-queue name="CSVUploadQMessageBean" entries="java:/jms/queue/CSV_UPLOAD_Q java:jboss/exported/jms/queue/CSV_UPLOAD_Q"/>
                <jms-queue name="InboundQMessageBean" entries="java:/jms/queue/CSV_INBOUND_Q java:jboss/exported/jms/queue/CSV_INBOUND_Q"/>
                <jms-queue name="OutboundQMessageBean" entries="java:/jms/queue/CSV_OUTBOUND_Q java:jboss/exported/jms/queue/CSV_OUTBOUND_Q"/>
                <jms-queue name="OutboundMessageBean" entries="java:/jms/queue/VFOREQUEST_Q java:jboss/exported/jms/queue/VFOREQUEST_Q"/>
                <jms-queue name="AutoManualInboundMessageBean" entries="java:/jms/queue/VFOAUTOMANUALRESPOSE_Q java:jboss/exported/jms/queue/VFOAUTOMANUALRESPOSE_Q"/>
                <jms-queue name="InboundMessageBean" entries="java:/jms/queue/VFORESPONSE_Q java:jboss/exported/jms/queue/VFORESPONSE_Q"/>
                <connection-factory name="InVmConnectionFactory" entries="java:/ConnectionFactory" connectors="in-vm"/>
                <connection-factory name="RemoteConnectionFactory" entries="java:jboss/exported/jms/RemoteConnectionFactory" connectors="http-connector" ha="true" block-on-acknowledge="true" reconnect-attempts="-1"/>
                <pooled-connection-factory name="activemq-ra" entries="java:/JmsXA java:jboss/DefaultJMSConnectionFactory" connectors="in-vm" transaction="xa"/>
            </server>
        </subsystem>
memorywildflycorretto


Solution 1:[1]

Had the same issue. This was the problem, because we moved to Azure which did not support multi-cast:

<broadcast-group name="bg-group1" jgroups-cluster="activemq-cluster" connectors="http-connector"/>

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1 P Dunkling

Related Questions