'Broken DAG: [...] No module named 'boto3'
I use docker-airflow from here with LocalExecutor, trying to connect to AWS with code below:
import datetime
import logging
from airflow import DAG
from airflow.contrib.hooks.aws_hook import AwsHook
from airflow.operators import (
PostgresOperator
)
import sql_statements
dag = DAG(
"dag_from_mrt_hourly",
start_date=datetime.datetime(2018, 1, 1, 0, 0, 0, 0),
end_date=datetime.datetime(2018, 12, 1, 0, 0, 0, 0),
schedule_interval="@monthly",
max_active_runs=1
)
...
But the error Broken DAG: [/usr/local/airflow/dags/dag_mrt_hourly.py] No module named 'boto3' showns in the UI.
Then I checked the ReadMe in Install custom python package section:
...or add it as a volume in docker-compose file...
Does it mean I can create a requirement.txt and ask docker-compose.yml to install the packages?
How to do that?
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
