'Unable to read CSV file in logstash as its shutting down on its own

I have elasticsearch and Kibana are up and running and I want to read logs using logstash so for that I have passed csv file as an input in logstash.conf file but its not reading logs and shutting down automatically.

This is how I am running logstash command:

D:\logstash-8.1.0\bin>logstash -f "D:/logstash.conf"

logstash.conf

input{
 file{
   path => "D:/unicorn.csv"
   start_position => beginning
 }
}
output{
  elasticsearch{
    hosts => "localhost:9200"
    index => "indexforlogstash"
}
stdout{}
}

Below are the terminal output:

 "Using bundled JDK: ."
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and 
will likely be removed in a future release.
Sending Logstash logs to D:/logstash-8.1.0/logs which is now configured via log4j2.properties
[2022-03-16T12:59:47,905][INFO ][logstash.runner          ] Log4j configuration path used is: 
D:\logstash-8.1.0\config\log4j2.properties
[2022-03-16T12:59:47,938][WARN ][logstash.runner          ] The use of JAVA_HOME has been d 
deprecated. Logstash 8.0 and later ignores JAVA_HOME and uses the bundled JDK. Running 
Logstash with the bundled JDK is recommended. The bundled JDK has been verified to work with 
each specific version of Logstash, and generally provides best performance and reliability. If 
you have compelling reasons for using your own JDK (organizational-specific compliance 
requirements, for example), you can configure LS_JAVA_HOME to use that version instead.
[2022-03-16T12:59:47,942][INFO ][logstash.runner          ] Starting Logstash 
{"logstash.version"=>"8.1.0", "jruby.version"=>"jruby 9.2.20.1 (2.5.8) 2021-11-30 2a2962fbd1 
OpenJDK 64-Bit Server VM 11.0.13+8 on 11.0.13+8 +indy +jit [mswin32-x86_64]"}
[2022-03-16T12:59:47,947][INFO ][logstash.runner          ] JVM bootstrap flags: [-Xms1g, - 
Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, - 
XX:+UseCMSInitiatingOccupancyOnly, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, - 
Djruby.compile.invokedynamic=true, -Djruby.jit.threshold=0, -Djruby.regexp.interruptible=true, 
-XX:+HeapDumpOnOutOfMemoryError, -Djava.security.egd=file:/dev/urandom, - 
Dlog4j2.isThreadContextMapInheritable=true, --add-opens=java.base/java.security=ALL-UNNAMED, -- 
add-opens=java.base/java.io=ALL-UNNAMED, --add-opens=java.base/java.nio.channels=ALL-UNNAMED, - 
-add-opens=java.base/sun.nio.ch=ALL-UNNAMED, --add-opens=java.management/sun.management=ALL- 
UNNAMED]
 [2022-03-16T12:59:48,058][INFO ][logstash.settings        ] Creating directory 
 {:setting=>"path.queue", :path=>"D:/logstash-8.1.0/data/queue"}
 [2022-03-16T12:59:48,104][INFO ][logstash.settings        ] Creating directory 
 {:setting=>"path.dead_letter_queue", :path=>"D:/logstash-8.1.0/data/dead_letter_queue"}
 [2022-03-16T12:59:48,285][WARN ][logstash.config.source.multilocal] Ignoring the 
 'pipelines.yml' file because modules or command line options are specified
 [2022-03-16T12:59:48,347][INFO ][logstash.agent           ] No persistent UUID file found. 
 Generating new UUID {:uuid=>"84410117-2fa7-499b-b55a-43a29192540e", :path=>"D:/logstash- 
8.1.0/data/uuid"}
 [2022-03-16T12:59:55,063][ERROR][logstash.config.sourceloader] No configuration found in the 
 configured sources.
 [2022-03-16T12:59:55,424][INFO ][logstash.agent           ] Successfully started Logstash API 
 endpoint {:port=>9600, :ssl_enabled=>false}
 [2022-03-16T13:00:00,591][INFO ][logstash.runner          ] Logstash shut down.
 [2022-03-16T13:00:00,609][FATAL][org.logstash.Logstash    ] Logstash stopped processing 
 because of an error: (SystemExit) exit
 org.jruby.exceptions.SystemExit: (SystemExit) exit
    at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:747) ~[jruby.jar:?]
    at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:710) ~[jruby.jar:?]
    at D_3a_.logstash_minus_8_dot_1_dot_0.lib.bootstrap.environment.<main>(D:\logstash- 
8.1.0\lib\bootstrap\environment.rb:94) ~[?:?]

Someone let me know what I am doing wrong.

logstashkibanaelkpostgresqlpostgresql-12


Solution 1:[1]

I believe the error is due to slash. in the config code

path => "D:\unicorn.csv"

also while executing

logstash -f "D:\logstash.conf"

I would also recommend to use this below command to check if there are any syntax errors in your code

logstash --config.test_and_exit -f "D:\logstash.conf"

Keep posted!!!Thanks!!!

Solution 2:[2]

In the output field you can add rubydebug to get to know the inputs whichever has been parsed logstash as an input . This will print the parsed data in command prompt

    output{stdout{
    codec => rubydebug
}

}

Solution 3:[3]

Error says that logstash is unable to locate configuration at location provided.

To solve the error follow below steps:

  1. Modify file path in input as follows
    input{
     file{
       path => "D:\\unicorn.csv"
       start_position => beginning
     }
    }
  1. Also run configuration with logstash as follows
D:\logstash-8.1.0\bin>logstash -f "D:\\logstash.conf"

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1 sudhagar ramesh
Solution 2
Solution 3 Bloomstar

Related Questions