Category "logstash"

How to retrieve and list the first element of a field use Elasticsearch query (two compare and find end deleted duplicated documents in same index)?

in my elasticsearch index all logs have a field called RES and the structure look like this : Number:"12131", amount:8, referenceNumber:"140102129728883", expir

Wildfly log4j2 logstash

I managed to use log4j2 with wildfly 26. I'm strugling to send the logs to logstash. I tried with gelf and socked appenders. <Configuration status="warn" nam

Logstash can't connect to Elasticsearch

I'm trying to connect to Elasticsearch from Logstash, but it fails due to certificate issues. To give some background data: Version 8.1.3 on Kubernetes with Ran

grok, parse iptables remote log

I am new with grok and haven't work with grok before, but now, I need a help. I have remote log message from remote system which contains iptables rules. May 11

I was Getting below error when i'm to import csv file using Logstash 8.1

I'm getting below error when I was executing below command in my cmd prompt C:\Program Files\Elk Stack\logstash-8.1.2\bin>logstash -f ./logstach.conf Logstac

Why is Elasticsearch 8x. starting interactively but not as a service Ubuntu 20.04?

The tip for fixing this issue on Ubuntu 16.04 does not seem to work. Here's a link to one of the areas of various tips: https://discuss.elastic.co/t/cant-start-

How to generate a logstash event within a custom java filter plugin after timeout?

I want to correlate messages sending to logstash within a filter and send them to the next filter of logstash inside the pipeline. However I have successfully s

Configure multiline from filebeat tomcat module

I'm using filebeat 7.15 with tomcat module to send logs to kibana. I'm trying to send exceptions as one message. My tomcat.yml configuration looks like this. -

Java log parsing with logstash grok

This is my sample java log I tried to parse using Logstash [#|2022-04-06T07:02:47.885+0800|INFO|sun-appserver2.1|javax.enterprise.system.stream.out|_ThreadID=24

Upserting documents in elasticsearch using multiple fields instead of one

Further to question Upsert documents in Elasticsearch using custom ID field, now I need to upsert the documents with 2+ fields from the input. Taking the same e

Send data to Logstash in docker-compose environment

I've setup the ELK stack (Elasticsearch, Logstash and Kibana) by following this Github repo. Now I want to send data to logstash, for example specifying a pipel

Elasticsearch&Logstash Log4j Vulnerabilities

We have log4j vulnerabilities for Elasticsearch and Logstash in the following paths: Path : /usr/share/Elasticsearch/lib/log4j-core-2.11.1.jar Path : /usr/share

How to detect logstash input connection error

How can I monitor and detect errors when connecting kafka to logstash. Say for example my kafka broker is down and no connection is established between kafka an

ELK stack error elastic search don't authorize Logstash

I followed up this blog to start ELK stack from docker compose file but used version 8.1.2. It is not running successfully elastic search don't authorize Logst

How to test logstash Marker in Junit

I have to create a Junit test for my class logging Map<String, Object> mapMarker = new HashMap<>(); mapMarker.put("input_source", "test_input_source

Can Kafka Connect consume data from a separate kerberized Kafka instance and then route to Splunk?

My pipeline is: Kerberized Kafka --> Logstash (hosted on a different server) --> Splunk. Can I replace the Logstash component with Kafka Connect? Could

logstash settings are invalid

I'am trying to start logstash on windows with a custom conf file but I get always the same error : Your settings are invalid. Reason: Setting "" doesn't exist.

Logstash with persistent queue

I have started logstash using following configurations: Inside logstash.yml: queue.type: persisted queue.max_bytes: 8gb queue.checkpoint.writes: 1 configura

elasticsearch es_rejected_execution_exception

I'm trying to index a 12mb log file which has 50,000 logs. After Indexing around 30,000 logs, I'm getting the following error [2018-04-17T05:52:48,254][INFO ]

Is it possible to configure multiple output for a filebeat?

In one of our applications we parse the application logs using logstash and indexing them into elasticsearch. Our simple architecture is logfiles ---> filebe