'How to calculate average data size in KB for kinesis stream?

I am trying to figure out the number of shards required for my kinesis stream. I am looking at IncomingBytes and IncomingRecords metric but can't seem to understand how to estimate the average size of the record written to the stream in kilobytes (KB).

I am using the following formula to calculate number of shards:

 number_of_shards = max (incoming_write_bandwidth_in_KB/1000, outgoing_read_bandwidth_in_KB/2000)

where

incoming_write_bandwidth_in_KB = average_data_size_in_KB multiplied by the number_of_records_per_seconds.

outgoing_read_bandwidth_in_KB = incoming_write_bandwidth_in_KB multiplied by the number_of_consumers.


Solution 1:[1]

As I see it the answer is very dependent on your team an the board members. Did you try to talk to your manager about the situation? maybe you need more board members.

Also, maybe if you can started working in agile with sprints of 2 weeks. then there will be time limit by the sprint borders end for every issue.

In my team each member can approve the issue of fellow members and it works well for as, you can see what others are doing and you have incentive to approve the issue in time, because the next time it will be your issue.

For more info on scrum, agile and sprints. https://www.atlassian.com/agile/scrum/sprints

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1 Dharman