'Broken DAG: [...] No module named 'boto3'

I use docker-airflow from here with LocalExecutor, trying to connect to AWS with code below:

import datetime
import logging

from airflow import DAG
from airflow.contrib.hooks.aws_hook import AwsHook

from airflow.operators import (
    PostgresOperator
)

import sql_statements

dag = DAG(
    "dag_from_mrt_hourly",
    start_date=datetime.datetime(2018, 1, 1, 0, 0, 0, 0),
    end_date=datetime.datetime(2018, 12, 1, 0, 0, 0, 0),
    schedule_interval="@monthly",
    max_active_runs=1
)

...

But the error Broken DAG: [/usr/local/airflow/dags/dag_mrt_hourly.py] No module named 'boto3' showns in the UI.

Then I checked the ReadMe in Install custom python package section:

...or add it as a volume in docker-compose file...

Does it mean I can create a requirement.txt and ask docker-compose.yml to install the packages?

How to do that?



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source