'Broken Airflow DAG: No Module Found From Docker Requirement Instillation
I am facing an issue where my DAG isn't importing into Airflow due to a "ModuleNotFoundError: No Module Name package" error.
The provider throwing the error shows in the Airflow UI and works if I import it inside of a Python Operator. I am installing the provider via a requirements file in a Docker Image which also shows correct instillation and show's it's installed in site packages. I am running via a Celery executor on Kubernetes. Could this have something to do with the issue?
Solution 1:[1]
This occurred due to several Kubernetes pods not being re-built. This caused my scheduler to not have the required packages which caused the import errors. I manually restarted those nodes and re-built it from the last Docker file and everything ran smoothly.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|---|
| Solution 1 | Kenneth Lockett |