'How to access file metadata, for files in google cloud storage, from a python google cloud function
I'm trying to access the custom metadata on a file in Google cloud storage from within a Cloud Function, but it always returns "None".
The file definitely has custom metadata on it - the custom metadata can be seen from the GCS browser.
I can access the custom metadata successfully if I run my code in a cloud shell environment.
But if I run the same code in a cloud function, then the returned blob.metadata is always "None".
What am I doing wrong? What do I have to do differently in a Cloud Function compared to a Cloud Shell.
from google.cloud import storage
client = storage.Client()
bucket = client.bucket(<my bucket name>)
blob = bucket.get_blob(<my filename>)
metadata = blob.metadata
Note also that blob.download_as_string() gets me the file contents correctly - it's just the custom metadata I can't get.
Solution 1:[1]
blob.metadata only returns Storage object's custom metadata (a dict). None means that there is no custom metadata. See the docs of metadata :
Retrieve arbitrary/application specific metadata for the object.
The documentation of Object resource (API) specify that metadata is :
User-provided metadata, in key/value pairs.
Note that custom metadata is different from fixed-key metadata, that you can also edit with Edit metadata button in Google Cloud Console. Fixed-key metadata contains :
- Content-Type
- Content-Encoding
- Content-Disposition
- Content-Language
- Cache-Control
This particular kind of metadata can be accessed via blob.content_type, blob.content_encoding, ... (check a complete example).
To add custom metadata, just click Add item button on the same window (Edit metadata) or use gsutil (see Editing object metadata docs) :
gsutil setmeta -h "x-goog-meta-examplekey:examplevalue" gs://<your-bucket>
Solution 2:[2]
In fact blob.metadata will not show the user metadata, you have to add blob.patch() and metadata will appear in blob.metadata variable
from google.cloud import storage
client = storage.Client()
bucket = client.bucket(<my bucket name>)
blob = bucket.get_blob(<my filename>)
blob.patch()
metadata = blob.metadata
To save user metadata, you can set blob.metadata and then call blob.patch to add or modify metadata, or blob.update to erase user metadata for this blob
Solution 3:[3]
Use blob.get_metadata, NOT blob.metadata
From the docs, blob.metadata does not make an HTTP request.
Solution 4:[4]
I ran into the same problem. But in my case, I have a trigger to run a cloud function whenever an object is created in a bucket. I needed to get the metadata as well using the code that you used, but it returns None as well.
I then changed the trigger to google.storage.object.metadataUpdate, so my Cloud Function would trigger whenever I set or update the metadata of an object. By doing that, I could get the metadata using the same code.
Solution 5:[5]
blob.reload() can be used to refresh metadata from upstream.
blob = storage.Blob("name", bucket=bucket)
blob.reload()
metadata: dict = blob.metadata
Reload properties from Cloud Storage.
https://googleapis.dev/python/storage/2.2.1/blobs.html#google.cloud.storage.blob.Blob.reload
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|---|
| Solution 1 | |
| Solution 2 | |
| Solution 3 | Sabito 錆兎 stands with Ukraine |
| Solution 4 | jdb |
| Solution 5 | klden |