'How can I set GOOGLE_APPLICATION_CREDENTIALS in AWS Glue?
I have an Glue script where I desire to use the GoogleCloud API py to manage the data of my BigQuery tables and combine them with the S3 catalog table.
I added to the script the Job parameter:
Key --additional-python-modules
Value s3://MYS3PATH/google_cloud_bigquery-2.34.0-py2.py3-none-any.whl
The code of my job uses:
from google.cloud import bigquery
path_s3_google_credential = "s3://MYS3PATH/service_key_google.json"
os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = path_s3_google_credential
project_id = "myproj-google-bigquery"
client_bq = bigquery.Client(project=project_id)
When I run the code, I receive:
DefaultCredentialsError: File s3://MYS3PATH/service_key_google.json was not found.
I also have in AWS Secrets Manager the service account already set with the same service key defined inside my service_key_google.json.
How can I proceed to allow my job to connect to BigQuery?
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|