'How to hide aws_key_id and aws_secret_key under Rendered Template in AWS MWAA
I am using AWS Managed Airflow (MWAA) with Airflow version 2.0.2. To setup External Tables in Snowflake I am using IAM User and I need to pass aws_key_id and aws_secret_key in Create Stage statements. I have templated my Create Stage statements and passing the aws_key_id and aws_secret_key by reading them from AWS Secrets Manager. Here's how my Create Stage statement is
CREATE OR REPLACE STAGE dev_stage.product_analytics.propsect_square_sftp_campaign_data0 URL='s3://rlg-eapedw-qa-curatedzone/nrtllc/curate_zone/campaign' credentials= (aws_key_id='{{task_instance.xcom_pull(task_ids='read_secrets_manager', key='step_data_1')}}' aws_secret_key='{{task_instance.xcom_pull(task_ids='read_secrets_manager', key='step_data_2')}}') file_format = (TYPE=PARQUET);
But I am finding that after the code runs successfully, the Renderer shows the aws_key_id and aws_secret_key. I am wondering how to avoid this. I tried using mask_secrets from airflow.utils.log.secrets_masker import mask_secret but looks like this is not supported in 2.0.2.
Wondering anyone has solved this problem on AWS.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
