'airflow - get task instance attributes
Could someone let me know how to get task instance attribute for task within a task. So I am calling a databricks job from my task and it takes a execution date as a parameter.
I tried following thing.
batch_date = "{{ execution_date }}"
batch_date = execution_date.split("T")[0]
spark_jar_task = DatabricksSubmitRunOperator(
task_id='run_ingestion_jar',
new_cluster=new_cluster,
spark_jar_task={'main_class_name': 'com.batchingestion.Main', "parameters":["915", batch_date]},
libraries=[{'jar': 'gs://aaa/test.jar'}]
)
This is partially working. Rather than getting batch_date like 2022-02-07 I am getting whole execution date 2022-02-07T21:13:00.979140+00:00
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
