Category "google-cloud-composer"

GCP-Cloud Composer: Secret Manager access variable.json

I try to configure Secret Manager for my Composer (ver 1.16, airflow 1.10) but I have a weird situation like below. In my Composer, I've used a variable.json fi

Airflow DAG fails when PythonOperator with error "Negsignal.SIGKILL"

I am running Airflowv1.10.15 on Cloud Composer v1.16.16. My DAG looks like this : from datetime import datetime, timedelta # imports from airflow import DAG fr

KubernetesPodOperator is not able to access the Airflow Connections stored in Google Secrets Manager

I am using the Composer version 2.0.0 Airflow 2.1.4 and I have created a KubernetesPodOperator that is trying to access the Airflow connection stored in the Goo

BigQueryInsertJobOperator dryRun is returning success instead of failure on composer (airflow)

When using BigQueryInsertJobOperator and setting the configuration to perform a dry run on a faulty .sql file/ a hardcoded query, the task succeeds even though

loop over airflow variables issue question

I am having hard time looping over an airflow variable in my script so I have a requirement to list all files prefixed by string in a bucket. next loop throug