'Is it possible to re-use a session between SnowflakeOperator's?
I have a DAG where I want to have multiple separate SnowflakeOperator's use the same snowflake SESSION. The reason being, I want them all to use the same TEMPORARY table's.
Unfortunately, even when using the same airflow snowflake connection, usage of session like CLIENT_SESSION_KEEP_ALIVE and heartbeat along with ensuring the default session timeout is set high enough, the operator always creates a brand new session with each operator.
This forces me to either pass all of my queries in a array to one operator (not always possible if i want to do something in the middle) OR to use TRANSIENT tables for which I need to make sure get cleaned up and that multiple instances of the run don't tread all over eachother.
Has anyone managed to do this?
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|