Cloud run custom domain returning upstream request timeout

Question

I have the same service deployed on 2 different GCP projects. In one project, it's working correctly while the other project is not working. In advance, the only differences are the environment variables listed on the project level - mainly the IP addresses of our data repositories.

Focusing on the project where it's not working, I have the service fully online, and when I hit the default URL generated by Cloud Run (https://xxxxxxxxx-xxxx-yyyyyyyy-uc.a.run.app), everything works perfectly. The /health endpoint created to test the data repositories connectivity returns a payload with ALL GOOD message, and no error is log printed, meaning everything is successfully running.

The problem occurs when I replace the default URL with any of our custom subdomains: prd-api.yyyyyy.com/health or api2.yyyyyy.com/health. All of these domains are mapped to the right service deployed more than 48hrs ago, the certificate seems to be issued, but the browser continuously hangs when I hit it. Only after long minutes of waiting returns::

upstream request timeout

What do logs tell me?

• Once again, the default URL (https://xxxxxxxxx-xxxx-yyyyyyyy-uc.a.run.app) I get no issue:

GET200 1.08 KB 15.7 s Chrome 96 https://xxxxxxxxx-xxxx-yyyyyyyy-uc.a.run.app/health

This means connections to all data repositories are successfully established.

• Compared with the custom domain, the log reports:

MongoError: failed to connect to server [mongo_ip_address:27017] on first connect [Error [MongoError]: connection 5 to mongo_ip_address:27017 timed out

It's important to say this service has a static outbound IP address which is whitelisted to allow access to the database. Otherwise, this wouldn't work with the default domain.

Answer 1

Based on your post, you are mapping to a specific URL path, /health.

As stated in the official documentation, Map a custom domain using Cloud Run domain mapping:

You can map a domain, such as example.com or a subdomain, such as subdomain.example.com. You can only map a domain to /, not to a specific URL path like /users.

EDIT:

Problem was with cookies, clearing cookies on browser solved the issue.