'BigQuery: Set expirationTime on Snapshot
I'm trying to create a BigQuery snapshot of a table with an expiration date 60 minutes in the future.
It looks like I can't define a expiration time when creating the snapshot, as CopyJobConfig doesn't support it.
So I create the snapshot first, then try to update it's metadata.
Using the following code
expDate = datetime.now() + timedelta(hours=1)
dstTable = client.get_table(f"{project}.{dataset}.{tblName}")
dstTable.expires = expDate
client.update_table(dstTable, ['expires'])
doesn't work as I get:
google.api_core.exceptions.BadRequest: 400 PATCH ... : Only table expirationTime, description, friendlyName and labels can be updated for a snapshot.
OK, so it looks like I need to use expirationTime for snapshots, so I tried:
expDate = datetime.now() + timedelta(hours=1)
dstTable = client.get_table(f"{project}.{dataset}.{tblName}")
dstTable.expirationTime= expDate
client.update_table(dstTable, ['expirationTime'])
Which kind of works, but the timestamp is set 6 months into the future, not 1 hour as I've specified:
There is no default expiration set on the dataset, so I don't understand where the 6 months are coming from. How can I properly set a snapshot's expiration time?
Solution 1:[1]
Snapshot expiration time can be set in BigQuery by using following four ways only :
- BigQuery Console
- SQL
- bq tool
- API
If the expiration time is not set then the snapshot will automatically set the expiration time after 6 months. For your requirement you can choose any one of the above options to set snapshot expiration time. For more information you can check this documentation. Snapshots metadata can be updated. You can keep track on this Issue Tracker for details on failing to update metadata of table snapshots.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
Solution | Source |
---|---|
Solution 1 | Shipra Sarkar |