How to set dynamic limits in Logstash throttle plugin based on the value of a field?

Question

I am using Logstash throttle plugin to limit the rate of events. Everything is ok if I set after_count manually. However I am trying to get this value from Redis. Then set it as a new field and then use it in throttle plugin. So my config file is something like this:

filter {
        ruby {
                init => 'require "redis"; $redis = Redis.new(host: "127.0.0.1", port: 6379, db: 5)'
                code => 'event.set("limit", $redis.get("limit"))'
        }

        mutate {
                convert => {
                        "limit" => "integer"
                }
        }

        throttle {
                after_count => %{limit}
                period => 1
                max_age => 2
                key => "[message]"
                add_tag => "throttled"
        }

        if "throttled" in [tags] {
                drop { }
        }
}

Although there is no error when checking logs, indexing events is stopped when running this config. Can anyone helps me? Is there anyway to limit the rate based on a value in redis?

Answer 1

No, you cannot use a sprintf reference in the before_count or after_count options of a throttle filter. The code uses the option values directly, it does not sprintf them.

"%{limit}" will be converted to zero, so the 'count > 0' always evaluates true, and every event is tagged and dropped.