'Copy Airflow project into Airflow DAGs directory
I have an Airflow project with the following structure
my_project
|
dags-custom-|custom_operator1.py
| |custom_operator2.py
| |custom_hook1.py
|
my_dag.py
Normally, if I have only my_dag.py I would do
cp my_dag.py ~/airflow/dags/
However, in this case I am not sure what should I do to load the project into Airflow DAGs list?
should I do cp ~/my_project/ ~/airflow/dags/?
Solution 1:[1]
Airflow tries to parse every python file as a DAG and if it finds that a python file has the DAG context, then it is loaded. In your case, you are correct, just copy over the project to airflow DAGs.
Assuming that you are using the docker image to run airflow, it would be advisable to mount your project's root into the airflow DAG root like this:
./my_project/:/opt/airflow/dags
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|---|
| Solution 1 | Pbd |
