'airflow webserver Workers terminating due to signal 11

I'm trying to start the airflow webserver with an existing application and the latest version (2.2.2). The executor is set as LocalExecutor, the datastore is Postgres, Python v3.9. Upon start up, it launches 4 workers which promptly die. It then spins its wheels continually trying to restart them.

Here's an example of the messages showing a worker starting and dying with signal 11 (segmentation violation). This is all within a second of starting.

Using worker: sync
[2021-11-30 17:29:31 -0500] [12529] [INFO] Booting worker with pid: 12529
[2021-11-30 17:29:31 -0500] [12530] [INFO] Booting worker with pid: 12530
[2021-11-30 17:29:31 -0500] [12531] [INFO] Booting worker with pid: 12531
[2021-11-30 17:29:31 -0500] [12532] [INFO] Booting worker with pid: 12532
Running the Gunicorn Server with:
Workers: 4 sync
Host: 0.0.0.0:8080
Timeout: 120

settings.prepare_engine_args(): Using pool settings. pool_size=5, max_overflow=10, pool_recycle=1800, pid=12529

[WARNING] Worker with pid 12529 was terminated due to signal 11

Any suggestions on how to debug these workers?

pythonmacosairflowapple-m1


Solution 1:[1]

I gave up on trying to run Airflow directly on my M1 Mac. I suspect it has something to do with the emulator for the M1 chip, but don't know for sure. I fell back to running Airflow in Docker. It takes a very long time to start, but does run OK that way.

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1 user2701620

Related Questions