GKE: Impossible to delete a cluster

Question

I have a weird issue with GKE, the cluster has been created by Terraform, and I tried to make a change requiring a deletion and re-creation. It failed at the re-creation because I was missing an API, so I added it and retry. Thing is that I have a cluster that exists, empty but with failed to delete cluster message on it.

I never had this issue and I already destroyed and re-created this very resource. I tried to destroy all the resources created by terraform on this project but I still get an error "failed to delete cluster".

Also I tried to do it by hand on the UI but still get the same error.

I tried to do it using gcloud container clusters delete <cluster_name> and got "Failed to delete cluster, name: operation-xxx-xxx..." and got a link to the operation failed.

It's a JSON with a 401 code, with the following message:

Request is missing required authentication credential. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.

I tried to re-auth but it doesn't help I get the same error.

I'm running out of idea, can you help me here?

Answer 1

A 401 (unauthorized) suggests that you've insufficient permissions to delete the cluster.

Either get a role that permits your user account to delete clusters.

Or ask someone who has an account that has sufficient powers to delete it for you.

Or authenticate gcloud (gcloud activate-service-account) with the Service Account that you used to create the cluster (assuming it can delete clusters too) and then use gcloud container clusters delete ... optionally include --account=${SERVICE_ACCOUNT_EMAIL} or just ensure the Service Account is ACTIVE with gcloud auth list.

Answer 2

I did not found a proper solution, but what did work was to delete the whole project and start over.

Luckily for me it was a lab, not a production project...