ASP.NET Core 3.1 with google signin, nginx reverse proxy and docker Resource temporarily unavailable

Question

I'm building an ASP.NET Core 3.1 web application with the goal to be hosted on an AWS EC2 instance.

My setup is as follows:

• EC2 instance running Debian 10
• Nginx reverse proxy
• certbot for ssl certificate
• Docker running in swarm mode
• ASP.NET Core 3.1 web application (docker)
• SQL Server database (docker)

The web application and SQL Server database are both running in containers and are described in a docker-compose.yml file:

version: '3.5'

services:
  db:
    image: mcr.microsoft.com/mssql/server:2017-latest
    deploy:
      replicas: 1
    environment:
      - ACCEPT_EULA=y
      - SA_PASSWORD=${SA_PASSWORD}
    ports:
      - 1433:1433
    volumes:
      -${MSSQL_STORAGE}:/var/opt/mssql

  ui:
    image: ${PRIVATE_DOCKER_REGISTRY}/${APPLICATION_NAME}/${SERVICE}:${TAG}
    deploy:
      replicas: 1
    ports:
    - 5000:80
    environment:
      ASPNETCORE_ENVIRONMENT: Development
      GOOGLE_CLIENT_ID: ${GOOGLE_CLIENT_ID}
      GOOGLE_SECRET_ID: ${GOOGLE_SECRET_ID}
      DB_HOST: db
      DB_NAME: ${DB_NAME}
      DB_PASSWORD: ${DB_PASSWORD}
      DB_USER: ${DB_USER}

My application when running in local debug mode and in a local docker swarm runs as expected (accessed with http://localhost:5000). I'm able to log in with my google authentication and I'm redirected to the correct page after login.

When I deploy to the ec2 instance however I'm unable to login.

docker service logs -tf {service_name} I see the following:

Microsoft.AspNetCore.Authentication.Google.GoogleHandler[4]

Error from RemoteAuthentication: Resource temporarily unavailable.

fail:
Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware[1]

An unhandled exception has occurred while executing the request. System.Exception: An error was encountered while handling the remote login.
System.Net.Http.HttpRequestException: Resource temporarily unavailable
System.Net.Sockets.SocketException (11): Resource temporarily unavailable

at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken)
--- End of inner exception stack trace ---
at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean allowHttp2, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.GetHttpConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.DiagnosticsHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts) at Microsoft.AspNetCore.Authentication.OAuth.OAuthHandler1.ExchangeCodeAsync(OAuthCodeExchangeContext context)
at Microsoft.AspNetCore.Authentication.OAuth.OAuthHandler1.HandleRemoteAuthenticateAsync() at Microsoft.AspNetCore.Authentication.RemoteAuthenticationHandler1.HandleRequestAsync()
--- End of inner exception stack trace ---
at Microsoft.AspNetCore.Authentication.RemoteAuthenticationHandler`1.HandleRequestAsync()
at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.Diagnostics.EntityFrameworkCore.MigrationsEndPointMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.Diagnostics.EntityFrameworkCore.DatabaseErrorPageMiddleware.Invoke(HttpContext httpContext)
at Microsoft.AspNetCore.Diagnostics.EntityFrameworkCore.DatabaseErrorPageMiddleware.Invoke(HttpContext httpContext)
at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)

My nginx configuration is as follows:

server {
        server_name mydomain.com;

        location / {
                proxy_pass         http://127.0.0.1:5000;
                proxy_http_version 1.1;
                proxy_set_header   Upgrade $http_upgrade;
                proxy_set_header   Connection keep-alive;
                proxy_set_header   Host $host;
                proxy_cache_bypass $http_upgrade;
                proxy_set_header   X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header   X-Forwarded-Proto $scheme;
        }


    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/mydomain.com/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/mydomain.com/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

}

I'm thinking maybe the reverse proxy configuration is incorrect somehow? Incorrect headers maybe or something else?

Answer 1

A common problem with Google Auth when your app is behind a reverse proxy is that the URL doesn't match the URL/domain name you supplied in the Google Developers Console. When using a reverse proxy, you are changing the request URL from mydomain.com to http://127.0.0.1:5000.

Adding the ForwardedHeaders in your code should fix this so the request keeps the original URL as well as the Forwarded Url

      // This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
        public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
        {
            app.UseForwardedHeaders(new ForwardedHeadersOptions
            {
                ForwardedHeaders = ForwardedHeaders.XForwardedProto
            });
            
           // other Configure code here