'How to not increment table versions in Glue when adding new Partitions
When adding new partitions into a table in Glue, I'm observing the versionId number of the table is increasing.
Since the number of versions per table is a limit in AWS(up to 100.000), I would like to control the situation by not removing versions using a script, instead, It would be great to have a behaviour like setting the table to not increment the version sequentially every time a new partition is added.
Is It possible to set a behaviour like that in AWS Glue? Even, set a fixed table version number for each partition would be great.
Note: I do not want to simply change the AWS limit, I mean, I want a permanent solution to this problem.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
