Reduce logging for retry_count in splunk

Question

I was trying to reduce the production of logs in splunk. when i tried to reduce the logs by using below query i was getting a large number of log entries

Spunk query: (source!="/var/log/kubernetes/audit/kube-apiserver-audit.log" cluster_name::dkp-test index="tst-dkp") null sourcetype="fluentd:monitor-agent"

I was tried to reduce the log production by using emit_interval 300 and cache_ttl 3600 but it is not worked.

Can anyone please suggest how should i reduce the production of logs in splunk connector. Thanks.

I found that the logs were producing from the node:

[ec2-user@ip-11-21-111-11 ~]$ sudo su
[root@ip-11-21-111-11 ec2-user]# ps -ef | grep flu
root     2536779 2536761  0 Jan31 ?        00:00:16 /usr/bin/fluentd -c /fluentd/etc/fluent.conf
root     2536965 2536779  0 Jan31 ?        00:07:34 /usr/bin/ruby -r/usr/local/share/gems/gems/bundler-2.2.33/lib/bundler/setup -Eascii-8bit:ascii-8bit /usr/bin/fluentd -c /fluentd/etc/fluent.conf --under-supervisor
root     2536978 2536965  0 Jan31 ?        00:00:00 sh -c jq  --unbuffered -c '.record.source = "namespace:platform/pod:splunk-connect-splunk-kubernetes-logging-vmkjx" | .record.sourcetype = "fluentd:monitor-agent" | .record.cluster_name = "platform-dkp-test" | .record.splunk_index = "ss-tst-dkp" | .record' 2>&1
root     2536980 2536978  0 Jan31 ?        00:00:02 jq --unbuffered -c .record.source = "namespace:platform/pod:splunk-connect-splunk-kubernetes-logging-vmkjx" | .record.sourcetype = "fluentd:monitor-agent" | .record.cluster_name = "platform-dkp-test" | .record.splunk_index = "ss-tst-dkp" | .record
root     3730152 3730072  0 13:21 pts/0    00:00:00 grep --color=auto flu

The logs are generating from the splunk are :

{ [-]
   emit_records: 0
   emit_size: 0
   output_plugin: false
   plugin_category: filter
   plugin_id: object:c11c
   retry_count: null
   type: jq_transformer
}
Show as raw text
host = ip-11-21-111-11.ec2.internalsource = namespace:platform/pod:splunk-connect-splunk-kubernetes-logging-jxsourcetype = fluentd:monitor-agent

in this i wanted reduce the logs for Reduce logging for retry_count: null

Answer 1

The only way to "reduce the production of logs" going into Splunk is to not log as much that the Universal Forwarder picks up, or that is sent via the HTTP Event Collector

If you want to filter events based on criteria in your SPL, then you need to look at the field(s) in question, and only select what you're looking for