Nginx docker container exits with error “fopen:No such file or directory:fopen('/etc/nginx/ssl/live/test.example.dev/fullchain.pem'” in Ubuntu 20.04

Question

Okay so I'm learning Docker and I am trying to deploy a test app with a subdomain (who's domain was bought from another provider) which is pointing to my server. The server already has non-dockerized Nginx setup which serves couple of other non-dockerized apps perfectly. And that part means Nginx is already using port 80 and 443. It's also worth mentioning that the subdomain's main domain (example.dev) has a non-dockerized app with active SSL cert from Let's Encrypt already running in the server. And now the subdomain (test.example.dev) somehow shows Nginx default page when visited. This is my server situation. Now let me explain what happens with Nginx and Certbot in a dockerized app.

The app is using 4 images to create 4 containers: Nodejs, Mongodb, Nginx and Certbot(for SSL). Before adding Certbot, I could perfectly access the app with :. But now I need to attach that subdomain (test.example.dev) to my app with Let's Encrypt SSL certificates.

So after the build is done with Docker Compose, I see that Nginx and Certbot is exited with errors.

This is my nginx/default.conf file:

server {
    listen 80;
    listen [::]:80;

    server_name test.example.dev;
    server_tokens off;

    location /.well-known/acme-challenge/ {
        root /var/www/certbot;
    }

    location / {
        return 301 https://test.example.dev$request_uri;
    }
}

server {
    listen 443 default_server ssl http2;
    listen [::]:443 ssl http2;

    server_name test.example.dev;

    ssl_certificate /etc/nginx/ssl/live/test.example.dev/fullchain.pem;
    ssl_certificate_key /etc/nginx/ssl/live/test.example.dev/privkey.pem;
    
    location /api {
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

        proxy_set_header Host $http_host;
        proxy_set_header X-NginX-Proxy true;
        proxy_pass http://practice-app:3050;
        proxy_redirect off;

    }
}

And here’s my docker-compose.yml file:

version: '3'
services:

  practice-app:
    build: 
      context: .
      args: 
        NODE_ENV: production
    environment:
      - NODE_ENV=production
    command: node index.js
    depends_on:
      - mongo
      
  nginx: 
    image: nginx:stable-alpine
    ports: 
      - "4088:80"
    volumes:
      - ./nginx/default.conf:/etc/nginx/conf.d/default.conf:ro
      - ./certbot/www:/var/www/certbot/:ro
      - ./certbot/conf/:/etc/nginx/ssl/:ro

  certbot:
    image: certbot/certbot:latest
    volumes:
      - ./certbot/www/:/var/www/certbot/:rw
      - ./certbot/conf/:/etc/letsencrypt/:rw
    depends_on:
      - nginx

  mongo:
    image: mongo:4.4.6
    environment:
      - MONGO_INITDB_ROOT_USERNAME=test
      - MONGO_INITDB_ROOT_PASSWORD=test
    volumes:
      - mongo-db:/data/db
      
volumes:
  mongo-db:

Nginx logs says:

/docker-entrypoint.sh: Configuration complete; ready for start up
2022/01/31 13:42:28 [emerg] 1#1: cannot load certificate "/etc/nginx/ssl/live/test.example.dev/fullchain.pem": BIO_new_file() failed (SSL: error:02001002:system library:fopen:No such file or directory:fopen('/etc/nginx/ssl/live/test.example.dev/fullchain.pem','r') error:2006D080:BIO routines:BIO_new_file:no such file)
nginx: [emerg] cannot load certificate "/etc/nginx/ssl/live/test.example.dev/fullchain.pem": BIO_new_file() failed (SSL: error:02001002:system library:fopen:No such file or directory:fopen('/etc/nginx/ssl/live/test.example.dev/fullchain.pem','r') error:2006D080:BIO routines:BIO_new_file:no such file)

And Certbot logs says:

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Certbot doesn't know how to automatically configure the web server on this system. However, it can still get a certificate for you. Please run "certbot certonly" to do so. You'll need to manually configure your web server to use the resulting certificate.

But after adding the following code:

command: certonly --webroot -w /var/www/certbot --force-renewal --email [email protected] -d test.example.dev --agree-tos

under certbot service, the log changed to this:

[17:00] [server1.com test] # docker logs test_certbot_1

Requesting a certificate for test.example.dev

Certbot failed to authenticate some domains (authenticator: webroot). The Certificate Authority reported these problems:
  Domain: test.example.dev
  Type:   unauthorized
  Detail: Invalid response from http://test.example.dev/.well-known/acme-challenge/HCFXwB1BXb-provr8lr6mJCDG9LRoGbVV0e9BWiiwAo [63.250.33.76]: "<html>\r\n<head><title>404 Not Found</title></head>\r\n<body>\r\n<center><h1>404 Not Found</h1></center>\r\n<hr><center>nginx</center>\r\n"

Hint: The Certificate Authority failed to download the temporary challenge files created by Certbot. Ensure that the listed domains serve their content from the provided --webroot-path/-w and that files created there can be downloaded from the internet.

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Some challenges have failed.
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.

What am I doing wrong here? Please give me a beginner friendly solution as I am new to DevOps.

Answer 1

You have some mistakes in your docker-compose file. Your nginx should be linked with Practice_app not on nginx and your practice app should open the port 3050 in here.

version: '3'
services:

  practice-app:
    build: 
      context: .
      args: 
        NODE_ENV: production
    environment:
      - NODE_ENV=production
    command: node index.js
    ports:
    - "3050:3050"
    depends_on:
      - mongo
      
  nginx: 
    image: nginx:stable-alpine
    ports: 
      - "4088:80"
    volumes:
      - ./nginx/default.conf:/etc/nginx/conf.d/default.conf:ro
      - ./certbot/www:/var/www/certbot/:ro
      - ./certbot/conf/:/etc/nginx/ssl/:ro
    links:
      - practice-app

  certbot:
    image: certbot/certbot:latest
    volumes:
      - ./certbot/www/:/var/www/certbot/:rw
      - ./certbot/conf/:/etc/letsencrypt/:rw
    depends_on:
      - nginx


  mongo:
    image: mongo:4.4.6
    environment:
      - MONGO_INITDB_ROOT_USERNAME=test
      - MONGO_INITDB_ROOT_PASSWORD=test
    volumes:
      - mongo-db:/data/db
      
volumes:
  mongo-db: