Extracting file path from filebeat into logstash and then into elastic index

Question

I'm trying to take a part of my path and give it as index to my elasticsearch index.

my logstash config file looks like this (Note: the config file might be wrong cuz I tried 100 different things)

# The # character at the beginning of a line indicates a comment. Use
# comments to describe your configuration.
input {
    beats {
            port => "5044"
        }
}
# The filter part of this file is commented out to indicate that it is
# optional.
filter {
  grok {
      match => ["[log][file][path]", "c:\\PL_Logs\\%{GREEDYDATA:index_name}\\%{GREEDYDATA}" ]
  }
}
#output {
# stdout { codec => rubydebug }
#}
output {
   elasticsearch {
        hosts => [ "localhost:9222" ]
        index => "%{index_name}-%{+YYYY.MM.dd}"
   }
    stdout { codec => rubydebug }
}

my logs are filed like this C:\Incoming_Logs\requested_part\somesubfolder_doesnt_matter\2022\03\18\2022031814.txt

my filebeat config is this:

filebeat.inputs:
- type: log
  enabled: true
  paths:
    - c:\Incoming_Logs\**\*.txt

My question is how can I take the requested_part part from my path and make it an index? I've tried many things like covering the index with [] (%[{index_name}]) I tried to match with "path" instead of "[log][file][path]" I've changed the grok condition to "c:\PL_Logs\%{GREEDYDATA:index_name}\**" and I've also tried to use "/" instead of double "\"

Can someone point me out what I'm doing wrong here?

Answer 1

I found the grok match that works for me from this question How to get parts of Filebeat source filename in Logstash

I made mine like this match => { "[log][file][path]" => ".*(\\|\/)(?<myIndex>.*)(\\|\/).*.*(\\|\/).*(\\|\/).*(\\|\/).*(\\|\/)" }

It works but I prefer a more elegance regex if someone can help.