'logback elasticsearch appender missing index in Elasticsearch

I have a Spring Boot application.

I want to send my logs directly from logback to elasticsearch. I'm using Logback Elasticsearch Appender (https://github.com/internetitem/logback-elasticsearch-appender)

The problem is that the index is not created in Elasticsearch. The File appender is working as expected and there are no errors in the logs or anything that shows a misconfiguration.

logback.xml:

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
    <appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
        <file>
            myApp.log
        </file>
        <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
            <fileNamePattern>
                myApp.%d{yyyy-MM-dd}.log
            </fileNamePattern>
            <maxHistory>
                15
            </maxHistory>
        </rollingPolicy>
        <encoder>
            <pattern>
                [%d] [%thread] [%-5level] [%class.%method] [user=%mdc{user:-n/d}] - %msg%n%ex{20}
            </pattern>
        </encoder>
    </appender>
  
    <appender name="ELASTIC" class="com.internetitem.logback.elasticsearch.ElasticsearchAppender">
        <url>http://localhost:9200/_bulk</url>
        <index>dev_myapp-%date{yyyy-MM-dd}</index>
        <type>logback</type>
        <connectTimeout>30000</connectTimeout> <!-- optional (in ms, default 30000) -->
        <errorsToStderr>false</errorsToStderr> <!-- optional (default false) -->
        <includeCallerData>false</includeCallerData> <!-- optional (default false) -->
        <logsToStderr>false</logsToStderr> <!-- optional (default false) -->
        <maxQueueSize>104857600</maxQueueSize> <!-- optional (default 104857600) -->
        <maxRetries>3</maxRetries> <!-- optional (default 3) -->
        <readTimeout>30000</readTimeout> <!-- optional (in ms, default 30000) -->
        <sleepTime>250</sleepTime> <!-- optional (in ms, default 250) -->
        <rawJsonMessage>false</rawJsonMessage> <!-- optional (default false) -->
        <includeMdc>false</includeMdc> <!-- optional (default false) -->
        <maxMessageSize>-1</maxMessageSize> <!-- optional (default -1 -->
<!--        <authentication class="com.internetitem.logback.elasticsearch.config.BasicAuthentication" />  optional -->
        <properties>
            <property>
                <name>severity</name>
                <value>%level</value>
            </property>
            <property>
                <name>thread</name>
                <value>%thread</value>
            </property>
            <property>
                <name>stacktrace</name>
                <value>%ex</value>
            </property>
            <property>
                <name>logger</name>
                <value>%logger{16}.%method</value>
            </property>
            <property>
                <name>server-time</name>
                <value>%date{ISO8601, America/Costa_Rica}</value>
            </property>
        </properties>
        <headers>
            <header>
                <name>Content-Type</name>
                <value>text/plain</value>
            </header>
        </headers>
    </appender>
 
    <root level="INFO">
        <appender-ref ref="FILE" />
        <appender-ref ref="ELASTIC" />
    </root>

    <logger name="com.myCompany.myApp" level="DEBUG" />
    <logger name="org.hibernate.hql" level="OFF"/>
    <logger name="com.zaxxer.hikari.HikariConfig" level="DEBUG"/>
    <logger name="com.zaxxer.hikari.pool.ProxyLeakTask" level="DEBUG"/>

</configuration>

I'd expect to see dev_myapp-2022-01-20 in my local Elasticsearch index, but it's not there.

Elasticsearch is running locally: curl localhost:9200

{
  "name" : "ubuntu-dev-stage-services",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "B0K_ThdDTlWftcVw23_NKA",
  "version" : {
    "number" : "7.16.3",
    "build_flavor" : "default",
    "build_type" : "deb",
    "build_hash" : "4e6e4eab2297e949ec994e688dad46290d018022",
    "build_date" : "2022-01-06T23:43:02.825887787Z",
    "build_snapshot" : false,
    "lucene_version" : "8.10.1",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

And this are the index in the instance: curl localhost:9200/_cat/indices

green open .geoip_databases                y1revQR7SyWHJIxnAq0z7w 1 0  42     0 40.3mb 40.3mb
green open .kibana_7.16.3_001              zjnnLKjXT362iQdFnXPsHQ 1 0 279    25  2.4mb  2.4mb
green open .apm-custom-link                t8-PvRWaTpSd1n2xuT8tYA 1 0   0     0   226b   226b
green open .apm-agent-configuration        hx8SsYzkTDqkEPpA7aZcvg 1 0   0     0   226b   226b
green open .kibana_task_manager_7.16.3_001 H-Kjao4hRLW45PmEMTvnGw 1 0  17 11520  1.6mb  1.6mb
green open .tasks                          Xp2ezFegRo-jzjK4UfUTpQ 1 0   6     0 28.9kb 28.9kb

EDIT

I was able to make it work by using Elasticsearch 5. I'm not sure but looks like the appender does not work with Elasticsearch 7



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source