Kubernetes AWX operator hanging on managed database migration

Question

I am running three Ubuntu VMs using VMware workstation and I have been able to succesfully deploy a 3 node Kubernetes cluster using k3sup along with k3s.

I am now trying to stand up AWX using the AWX operator but when I check the logs in the AWX pods I see the recurring messages.

I am not sure what I am doing wrong if maybe it has to do with routing or if I failed to open up a certain port on one of the nodes.

[wait-for-migrations] Waiting for database migrations...
[wait-for-migrations] Attempt 1 of 30
[wait-for-migrations] Waiting 0.5 seconds before next attempt
[wait-for-migrations] Attempt 2 of 30
[wait-for-migrations] Waiting 1 seconds before next attempt
[wait-for-migrations] Attempt 3 of 30
[wait-for-migrations] Waiting 2 seconds before next attempt
[wait-for-migrations] Attempt 4 of 30
[wait-for-migrations] Waiting 4 seconds before next attempt
[wait-for-migrations] Attempt 5 of 30
[wait-for-migrations] Waiting 8 seconds before next attempt
[wait-for-migrations] Attempt 6 of 30
[wait-for-migrations] Waiting 16 seconds before next attempt
[wait-for-migrations] Attempt 7 of 30

Answer 1

By default, database upgrades are not applied when upgrading the AWX image.

You must therefore manually connect to your AWX Web pod and run the following command:

awx-manage migrate --noinput

This should unblock the application startup.

Source: https://github.com/ansible/awx/issues/6539#issuecomment-609106337