How to set a cycle for an aws kinesis timestream

Question

i have a kinesis timestream which waits for newly created data in my s3-bucket so it can send it to my telegraf agent.

My question now would be if i can set a time in which my kinesis takes all the data which is created in time x and sends it summarized to my agent?

Now my timestream is running 24/7 therfore it costs me 1.15$ a day. Is there some kind of way to only stream every 12 hours and get all the data created in this time so the cost per day will only be 0,048 $ x 2 ?

Greetings Wig

Answer 1

As far as I understand from your problem, I can suggest using the lambda function to get the data from your S3 bucket and post it to your agent. Usually, Kinesis/Kafka streaming behind the polling mechanism, so it would be always looking for the change in data and stream through it.

In your specific case, You can use Cloudwatch event bridge as a scheduler and lambda function as a worker to manage the streaming. Here I'm attaching a few links for your reference.

I'm suggesting this because your main concern is about cost, hope this helps

https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-what-is.html