'Update: Fluent bit parsing JSON log as a text

I'm sending logs to ES with fluentd. App logs are in JSON format. Here is one of them

{"@timestamp":"2022-05-06T06:02:10.669Z", "log.level": "INFO", "message":"INFO Health check ok", "ecs.version": "1.2.0","service.name":"spring-boot-application","event.dataset":"spring-boot-application","process.thread.name":"http-nio-9001-exec-7","log.logger":"com.app.designer.rules.controller.RulesController","transaction.id":"15e36b5ef6cc69dd","trace.id":"fd275718e25c061d309954773985b101"}

In Kibana, i can see the logs enter image description here

JSON log is a value of log key. But, we want JSON Log key value, as Field and Value Please suggest. fluent bit config map is:

apiVersion: v1
kind: ConfigMap
metadata:
  name: fluent-bit-designer
data:
  fluent-bit-service.conf: |
    [SERVICE]
        Flush        1
        Daemon       Off
        Log_Level    error
        Parsers_File parsers.conf

  fluent-bit-input.conf: |
    [INPUT]
        Name              tail
        Path              /app/logs/designer-logs.log.json
        Multiline         On
        Parser_Firstline  multiline_pattern
        Refresh_Interval  5
        Mem_Buf_Limit     5MB
        Skip_Long_Lines   On
        Path_key          FileName

  fluent-bit-filter.conf: |
    [FILTER]
        Name    modify
        Match   *
        Add     PodName ${POD_NAME}
        Add     Namespace ${NAMESPACE_NAME}

  fluent-bit-output.conf: |
    [OUTPUT]   
       Name    forward
       Match   *
       Host    {{ .Values.efkCluster }}-fluentd-aggregator.{{ .Values.efkCluster }}
       Port    24224


  fluent-bit.conf: |
    @INCLUDE fluent-bit-service.conf
    @INCLUDE fluent-bit-input.conf
    @INCLUDE fluent-bit-filter.conf
    @INCLUDE fluent-bit-output.conf

  parsers.conf: |
    [PARSER]
        Name         multiline_pattern
        Format       regex
        Regex        ^\[(?<LogDate>\d{2}\-\d{2}\-\d{2} \d{2}\:\d{2}\:\d{2}\.\d{3}\+\d{4})\] (?<LogLevel>[^ ]*) * (?<Thread>\S+) (?<Username>[^\s]*?) (?<Logger>[A-Za-z0-9$_.]+) -(\r\n|\r|\n)*(?<Message>.*)
        time_key     LogDate
        time_format  %d-%m-%y %H:%M:%S.%L%z


Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source