'Why can't my airflow webserver initialize properly?

When I create a dummy DAG following the Apache guide to airflow in Docker and run docker-compose up, the webserver container repeatedly fails and restarts with errors that look like this:

webserver_1  | [2021-07-26 18:44:42,905] {cli_action_loggers.py:105} WARNING - Failed to log action with (psycopg2.errors.UndefinedTable) relation "log" does not exist
webserver_1  | LINE 1: INSERT INTO log (dttm, dag_id, task_id, event, execution_dat...

This error suggests to me that the webserver is unable to initialize the airflow db. This dummy DAG uses Airflow 2 and CeleryExecutor. I have run into the same issue following this example using Airflow 2 and LocalExecutor, as well as my team's current setup, which uses Airflow 1 and CeleryExecutor.

What confuses me is that my teammates can run DAGs based on our docker-compose without issue (and presumably, others can run Apache's official quickstart successfully). So something is wrong with my machine (likely related to Postgres) that is somehow affecting my Docker images. What could cause this error across several different docker-compose airflow setups?

What I have tried so far:

  • Restarting my computer
  • Killing, removing, and rebuilding my Docker images
  • Reinstalling Docker for Mac
  • Uninstalling and reinstalling libpq, postgres, and psycopg2 on my local machine (though in theory this shouldn't matter for the Docker environment?)

I am on Docker version 20.10.7, and docker-compose version 1.29.2.

postgresqldockerdocker-composeairflowpsycopg2


Solution 1:[1]

I was able to solve this problem by completely cleaning my computer of Docker artifacts using docker system prune. I ran the following commands:

docker system prune --all
docker system prune --all --volumes

My previous attempts to clear Docker artifacts from my computer had been incomplete; killing and removing containers and uninstalling/reinstalling Docker was not sufficient.

Solution 2:[2]

I believe you did not follow the "Before you begin" steps and increased memory needed for Docker https://airflow.apache.org/docs/apache-airflow/stable/start/docker.html#before-you-begin

Default settings for Docker for Mac, does not give you nearly enough memory for Docker to run Airflow.

Solution 3:[3]

I also have this issue if someone can help

airflow-init_1       | [2022-04-14 18:48:52,290] {cli_action_loggers.py:105} WARNING - Failed to log action with (psycopg2.errors.UndefinedTable) relation "log" does not exist
airflow-init_1       | LINE 1: INSERT INTO log (dttm, dag_id, task_id, event, execution_dat...

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1 Eli Sander
Solution 2 Jarek Potiuk
Solution 3 Canadian Lonman

Related Questions