'ActiveMQ Artemis cluster is not sending retained messages to new subscribers on different nodes

I am running two ActiveMQ Artemis 2.20.0 in a cluster. When publishing a message with retained set to true on broker1 clients that subscribe to the topic have the retained message delivered.

When clients subscribe to the topic on broker2 the retained message is not delivered. However, newly published messages are delivered to subscribers.

broker.xml (IP address are changed for each broker):

<configuration xmlns="urn:activemq"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xmlns:xi="http://www.w3.org/2001/XInclude"
        xsi:schemaLocation="urn:activemq /schema/artemis-configuration.xsd">

    <core xmlns="urn:activemq:core" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
            xsi:schemaLocation="urn:activemq:core ">

        <name>Broker1</name>

        <persistence-enabled>true</persistence-enabled>
        <journal-type>ASYNCIO</journal-type>
        <paging-directory>data/paging</paging-directory>
        <bindings-directory>data/bindings</bindings-directory>
        <journal-directory>data/journal</journal-directory>
        <large-messages-directory>data/large-messages</large-messages-directory>
        <journal-retention-directory period="7" unit="DAYS" storage-limit="10G">data/retention</journal-retention-directory>
        <journal-datasync>true</journal-datasync>
        <journal-min-files>2</journal-min-files>
        <journal-pool-files>10</journal-pool-files>
        <journal-device-block-size>4096</journal-device-block-size>
        <journal-file-size>10M</journal-file-size>
        <journal-device-block-size>4096</journal-device-block-size>
        <journal-file-size>10M</journal-file-size>
        <journal-buffer-timeout>588000</journal-buffer-timeout>
        <journal-max-io>4096</journal-max-io>
        <disk-scan-period>5000</disk-scan-period><!-- how often we are looking for how many bytes are being used on the disk in ms -->
        <max-disk-usage>90</max-disk-usage><!--once the disk hits this limit the system will block, or close the connection in certain protocols that won't support flow control. -->
        <critical-analyzer>true</critical-analyzer><!-- should the broker detect dead locks and other issues -->
        <critical-analyzer-timeout>120000</critical-analyzer-timeout>
        <critical-analyzer-check-period>60000</critical-analyzer-check-period>
        <critical-analyzer-policy>HALT</critical-analyzer-policy>
        <page-sync-timeout>34248000</page-sync-timeout>

        <connectors>
            <connector name="artemis">tcp://192.168.100.7:61616</connector> 
        </connectors>

        <acceptors>
            <acceptor name="artemis">tcp://192.168.100.7:61616</acceptor>
            <acceptor name="mqtt">tcp://192.168.100.7:1883?tcpSendBufferSize=1048576;tcpReceiveBufferSize=1048576;protocols=MQTT;useEpoll=true</acceptor>
        </acceptors>

        <broadcast-groups>
            <broadcast-group name="bg-group-1">
                <local-bind-address>192.168.100.7</local-bind-address>
                <local-bind-port>5432</local-bind-port>
                <group-address>231.7.7.7</group-address>
                <group-port>9876</group-port>
                <broadcast-period>2000</broadcast-period>
                <connector-ref>artemis</connector-ref>
            </broadcast-group>
        </broadcast-groups>

        <discovery-groups>
            <discovery-group name="dg-group-1">
                    <local-bind-address>192.168.100.7</local-bind-address>
                    <group-address>231.7.7.7</group-address>
                <group-port>9876</group-port>
                <refresh-timeout>10000</refresh-timeout>
            </discovery-group>
        </discovery-groups>

        <cluster-connections>
            <cluster-connection name="my-cluster">
                <address></address>
                <connector-ref>artemis</connector-ref>
                <check-period>1000</check-period>
                <connection-ttl>5000</connection-ttl>
                <min-large-message-size>50000</min-large-message-size>
                <call-timeout>5000</call-timeout>
                <retry-interval>500</retry-interval>
                <retry-interval-multiplier>1.0</retry-interval-multiplier>
                <max-retry-interval>5000</max-retry-interval>
                <initial-connect-attempts>-1</initial-connect-attempts>
                <reconnect-attempts>-1</reconnect-attempts>
                <use-duplicate-detection>true</use-duplicate-detection>
                <message-load-balancing>ON_DEMAND</message-load-balancing>
                <max-hops>1</max-hops>
                <confirmation-window-size>32000</confirmation-window-size>
                <call-failover-timeout>30000</call-failover-timeout>
                <notification-interval>1000</notification-interval>
                <notification-attempts>2</notification-attempts>
                <discovery-group-ref discovery-group-name="dg-group-1"/>
            </cluster-connection>
        </cluster-connections>

        <security-settings>
            <security-setting match="#">
                <permission type="createNonDurableQueue" roles="amq"/>
                <permission type="deleteNonDurableQueue" roles="amq"/>
                <permission type="createDurableQueue" roles="amq"/>
                <permission type="deleteDurableQueue" roles="amq"/>
                <permission type="createAddress" roles="amq"/>
                <permission type="deleteAddress" roles="amq"/>
                <permission type="consume" roles="amq"/>
                <permission type="browse" roles="amq"/>
                <permission type="send" roles="amq"/>
                <!-- we need this otherwise ./artemis data imp wouldn't work -->
                <permission type="manage" roles="amq"/>
            </security-setting>
        </security-settings>

        <address-settings>
            <!-- if you define auto-create on certain queues, management has to be auto-create -->
            <address-setting match="activemq.management#">
                <dead-letter-address>DLQ</dead-letter-address>
                <expiry-address>ExpiryQueue</expiry-address>
                <redelivery-delay>0</redelivery-delay>
                <max-size-bytes>-1</max-size-bytes>
                <message-counter-history-day-limit>10</message-counter-history-day-limit>
                <address-full-policy>PAGE</address-full-policy>
                <auto-create-queues>true</auto-create-queues>
                <auto-create-addresses>true</auto-create-addresses>
                <auto-create-jms-queues>true</auto-create-jms-queues>
                <auto-create-jms-topics>true</auto-create-jms-topics>
            </address-setting>
            <!--default for catch all-->
            <address-setting match="#">
                <dead-letter-address>DLQ</dead-letter-address>
                <expiry-address>ExpiryQueue</expiry-address>
                <redelivery-delay>0</redelivery-delay>
                <!-- with -1 only the global-max-size is in use for limiting -->
                <max-size-bytes>-1</max-size-bytes>
                <message-counter-history-day-limit>10</message-counter-history-day-limit>
                <address-full-policy>PAGE</address-full-policy>
                <auto-create-queues>true</auto-create-queues>
                <auto-create-addresses>true</auto-create-addresses>
                <auto-create-jms-queues>true</auto-create-jms-queues>
                <auto-create-jms-topics>true</auto-create-jms-topics>
                <auto-delete-queues>false</auto-delete-queues>
                <auto-delete-addresses>false</auto-delete-addresses>
                <default-address-routing-type>MULTICAST</default-address-routing-type>
                <redistribution-delay>0</redistribution-delay>
            </address-setting>
        </address-settings>

        <addresses>
            <address name="DLQ">
                <anycast>
                <queue name="DLQ" />
                </anycast>
            </address>
            <address name="ExpiryQueue">
                <anycast>
                <queue name="ExpiryQueue" />
                </anycast>
            </address>
        </addresses>

        <broker-plugins>
            <broker-plugin class-name="org.apache.activemq.artemis.core.server.plugin.impl.LoggingActiveMQServerPlugin">
                <property key="LOG_ALL_EVENTS" value="true"/>
                <property key="LOG_CONNECTION_EVENTS" value="true"/>
                <property key="LOG_SESSION_EVENTS" value="true"/>
                <property key="LOG_CONSUMER_EVENTS" value="true"/>
                <property key="LOG_DELIVERING_EVENTS" value="true"/>
                <property key="LOG_SENDING_EVENTS" value="true"/>
                <property key="LOG_INTERNAL_EVENTS" value="true"/>
            </broker-plugin>
        </broker-plugins>
    </core>
</configuration>

Broker logs when the retained message is published.

Broker-1. The broker messages are published to.

INFO  [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841012: delivered message with message ID: 11511, to consumer on address: activemq.notifications, queue: notif.b2a6b331-9952-11ec-994c-ac1f6bf1db84.ActiveMQServerImpl_name=Broker2, consumer sessionID: b2a5a1c0-9952-11ec-994c-ac1f6bf1db84, consumerID: 0
INFO  [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841010: routed message with ID: 11511, result: OK
INFO  [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841002: created session name: d3cdcb05-9952-11ec-8d06-ac1f6bfcbc48, session connectionID: dbd13123
INFO  [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841009: sent message with ID: 11514, session name: d3cd55d4-9952-11ec-8d06-ac1f6bfcbc48, session connectionID: dbd13123, result: OK
INFO  [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841014: acknowledged message ID: 11503, messageRef sessionID: UNAVAILABLE, with messageRef consumerID: UNAVAILABLE, messageRef QueueName: $sys.mqtt.retain.temp,  with ackReason: KILLED
INFO  [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841014: acknowledged message ID: 11498, messageRef sessionID: UNAVAILABLE, with messageRef consumerID: UNAVAILABLE, messageRef QueueName: $sys.mqtt.retain.temp,  with ackReason: KILLED

Broker2

INFO  [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841010: routed message with ID: 8201, result: NO_BINDINGS
INFO  [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841010: routed message with ID: 8202, result: NO_BINDINGS

mqtt activemq-artemis

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution	Source

'ActiveMQ Artemis cluster is not sending retained messages to new subscribers on different nodes

Sources

Related Questions