'How to dissect a log file with Filebeat that has multiple patterns?
I have trouble dissecting my log file due to it having a mixed structure therefore I'm unable to extract meaningful data.
A few example lines from my log:
2021.04.21 00:00:00.843 INF getBaseData: UserName = 'some username', Password = 'some pass', HTTPS=0
2021.04.21 00:00:00.843 INF getBaseData: UserName = 'some username', Password = 'some pass', HTTPS=0
2021.04.21 00:00:00.843 INF getBaseData: UserName = 'some username', Password = 'some pass', HTTPS=0
2021.04.21 00:00:00.858 INF *** BEGINNING OF ARCCore.performARCTask ***
2021.04.21 00:00:00.858 INF *** BEGINNING OF ARCCore.ProcessTask ***
2021.04.21 00:00:01.266 INF ARCCore.DCI4ARCSyncLogin: login successfully executed. - No error - DCI4ARCSync-CurrSessions/MaxSessions=17/400 CurrProcesses/MaxProcesses=16/250
2021.04.21 00:00:01.297 INF ARCCore.DCI4ARCSyncLogin: login successfully executed. - No error - DCI4ARCSync-CurrSessions/MaxSessions=7/400 CurrProcesses/MaxProcesses=7/250
2021.04.21 00:00:08.165 INF *** BEGINNING OF SYNC ARC TO DC ***--->bIsExternal:0
2021.04.21 00:00:08.434 INF BOC login successfully executed. - No Error - DCI4ARC-CurrSessions/MaxSessions=24/400 CurrProcesses/MaxProcesses=15/250
2021.04.21 00:00:08.635 INF BOCVersionNr ==> V16.1.00.00
2021.04.21 00:00:08.804 INF setEntitySnapshot successfully executed
2021.04.21 00:00:09.453 INF getSnapshotList successfully executed
2021.04.21 00:00:09.461 INF getARCVersion: ARCVersionNr ==> V16.0.00.06
I wrote a tokenizer with which I successfully dissected the first three lines of my log due to them matching the pattern but fail to read the rest.
My tokenizer pattern:
%{+timestamp} %{+timestamp} %{type} %{msg}: UserName = %{userName}, Password = %{password}, HTTPS=%{https}
the lines that get read successfully:
2021.04.21 00:00:00.843 INF getBaseData: UserName = 'some username', Password = 'some password', HTTPS=0 2021.04.21 00:00:00.843 INF getBaseData: UserName = 'some username', Password = 'some password', HTTPS=0 2021.04.21 00:00:00.843 INF getBaseData: UserName = 'some username ', Password = 'some password', HTTPS=0
The result:
{
"https": "0",
"msg": "getBaseData",
"password": "'20213197'",
"timestamp": "2021.04.21 00:00:00.843",
"type": "INF",
"userName": "'ARC_412_028_01_V01_2021042100000082'"
}
{
"https": "0",
"msg": "getBaseData",
"password": "'20213205'",
"timestamp": "2021.04.21 00:00:00.843",
"type": "INF",
"userName": "'ARC_412_028_01_V01_2021042100000084'"
}
{
"https": "0",
"msg": "getBaseData",
"password": "'20213205'",
"timestamp": "2021.04.21 00:00:00.843",
"type": "INF",
"userName": "'ARC_412_028_01_V01_2021042100000084'"
}
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
