'Unable to send Error level logs to elasticsearch
I am new to fluentd and I am using 2 .conf file 1 for aggregator and 1 for forwarder. I want to send only error level logs to elasticsearch but even after using the @loglevel error. It still showing me info level logs to elasticsearch. I am adding the conf files as there can be misconfiguration in the conf files.
----> aggregator.conf <------
<match fluent.**>
@type null
</match>
<source>
@type http
port 9880
</source>
<match fluentd.healthcheck>
@type stdout
</match>
<source>
@type tail
@id in_tail_container_logs
path /var/log/containers/*.log
exclude_path ["/var/log/containers/fluentd*"]
pos_file /var/log/fluentd-containers.log.pos
tag kubernetes.*
read_from_head true
format /^(?<time>.+) (?<stream>stdout|stderr)( (?<logtag>.))? (?<message>.*)$/
time_format %Y-%m-%dT%H:%M:%S.%L%z
</source>
<match **>
@type forward
<server>
host fluentd-0.fluentd-headless.infra-fluentd.svc.cluster.local
port 24224
</server>
<buffer>
@type file
path /opt/bitnami/fluentd/logs/buffers/logs.buffer
flush_thread_count 2
flush_interval 5s
</buffer>
</match>
-----> forwarder.conf <-----
<source>
@type prometheus
port 24231
</source>
<source>
@type prometheus_monitor
<labels>
host ${hostname}
</labels>
</source>
<source>
@type prometheus_output_monitor
<labels>
host ${hostname}
</labels>
</source>
<match fluent.**>
@type null
</match>
<source>
@type forward
bind 0.0.0.0
port 24224
</source>
<source>
@type http
bind 0.0.0.0
port 9880
</source>
<match fluentd.healthcheck>
@type stdout
</match>
<filter kubernetes.**>
@type kubernetes_metadata
</filter>
<match **>
@type elasticsearch
include_tag_key true
host "#{ENV['ELASTICSEARCH_HOST']}"
port "#{ENV['ELASTICSEARCH_PORT']}"
logstash_format true
<buffer>
@type file
path /opt/bitnami/fluentd/logs/buffers/logs.buffer
flush_thread_count 2
flush_interval 5s
</buffer>
</match>
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
