Why doesn't certbot-dns-cloudflare plugin install under aws linux

Question

I have an elastic beanstalk with a single ec2 instance and I need to install an SSL certificate during deployment and at this time the server can't be reached via the ip address given by the A record on the DNS. I would like to use LetsEncrypt with the certbot-dns-cloudflare plugin to automatically get and install a certificate. I have created a cloudflare credentials file containing my cloudflare api key so that the plugin can request cloudflare to create a DNS TXT record and use it to do the domain name ownership validation.

I encountered a number of problems when attempting to install certbot using the method described here https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/SSL-on-an-instance.html#letsencrypt (EPEL libraries not containing certbot), and appeared to have better luck using the cerbot-auto install method here https://medium.com/@mohan08p/install-and-renew-lets-encrypt-ssl-on-amazon-ami-6d3e0a61693.

So my process so far is:

$ wget https://dl.eff.org/certbot-auto
$ chmod a+x certbot-auto
$ sudo ./certbot-auto --debug --install-only 

This appears to get certbot installed and I see no error messages. Next I do this:

$ cd /opt/eff.org/certbot/venv
$ source bin/activate
$ sudo pip install certbot-dns-cloudflare    

    ... cut short for brevity ...
    Collecting zope.event (from zope.component->certbot>=0.21.1->certbot-dns-cloudflare)
      Downloading https://files.pythonhosted.org/packages/c5/96/361edb421a077a4c208b4a5c212737d78ae03ce67fbbcd01621c49f332d1/zope.event-4.4-py2.py3-none-any.whl
    Collecting pycparser (from cffi!=1.11.3,>=1.7->cryptography>=0.8->acme>=0.21.1->certbot-dns-cloudflare)
      Downloading https://files.pythonhosted.org/packages/68/9e/49196946aee219aead1290e00d1e7fdeab8567783e83e1b9ab5585e6206a/pycparser-2.19.tar.gz (158kB)
        100% |################################| 163kB 7.9MB/s
    Collecting zope.proxy (from zope.deferredimport>=4.2.1->zope.component->certbot>=0.21.1->certbot-dns-cloudflare)
      Downloading https://files.pythonhosted.org/packages/7c/f5/e9ed65cdf8c93d24d7512ef89e21b241bc9ae75d90bc8608cc142f4c26f9/zope.proxy-4.3.1.tar.gz (43kB)
        100% |################################| 51kB 12.1MB/s
    Installing collected packages: funcsigs, pbr, six, mock, zope.interface, chardet, idna, certifi, urllib3, asn1crypto, enum34, pycparser, cffi, ipaddress, cryptography, PyOpenSSL, requests, requests-toolbelt, pytz, pyrfc3339, josepy, acme, future, parsedatetime, ConfigArgParse, zope.hookable, zope.proxy, zope.deferredimport, zope.deprecation, zope.event, zope.component, certbot, jsonlines, cloudflare, certbot-dns-cloudflare
      Found existing installation: six 1.8.0
        Uninstalling six-1.8.0:
          Successfully uninstalled six-1.8.0
      Found existing installation: chardet 2.0.1
        DEPRECATION: Uninstalling a distutils installed project (chardet) has been deprecated and will be removed in a future version. This is due to the fact that uninstalling a distutils project will only partially uninstall the project.
        Uninstalling chardet-2.0.1:
          Successfully uninstalled chardet-2.0.1
      Found existing installation: urllib3 1.8.2
        Uninstalling urllib3-1.8.2:
          Successfully uninstalled urllib3-1.8.2
      Running setup.py install for pycparser ... done
      Found existing installation: requests 1.2.3
        Uninstalling requests-1.2.3:
          Successfully uninstalled requests-1.2.3
      Running setup.py install for future ... done
      Running setup.py install for ConfigArgParse ... done
      Running setup.py install for zope.hookable ... done
      Running setup.py install for zope.proxy ... done
      Running setup.py install for cloudflare ... done
    Successfully installed ConfigArgParse-0.13.0 PyOpenSSL-18.0.0 acme-0.29.1 asn1crypto-0.24.0 certbot-0.29.1 certbot-dns-cloudflare-0.29.1 certifi-2018.11.29 cffi-1.11.5 chardet-3.0.4 cloudflare-2.1.0 cryptography-2.4.2 enum34-1.1.6 funcsigs-1.0.2 future-0.17.1 idna-2.8 ipaddress-1.0.22 josepy-1.1.0 jsonlines-1.2.0 mock-2.0.0 parsedatetime-2.4 pbr-5.1.1 pycparser-2.19 pyrfc3339-1.1 pytz-2018.7 requests-2.21.0 requests-toolbelt-0.8.0 six-1.12.0 urllib3-1.24.1 zope.component-4.5 zope.deferredimport-4.3 zope.deprecation-4.4.0 zope.event-4.4 zope.hookable-4.2.0 zope.interface-4.6.0 zope.proxy-4.3.1
    You are using pip version 9.0.3, however version 18.1 is available.
    You should consider upgrading via the 'pip install --upgrade pip' command.

In the listing I see indications that the cloudflare plugin was successfully installed. However, when I list the plugins I don't see it:

$ sudo ./certbot-auto plugins
Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
* apache
Description: Apache Web Server plugin
Interfaces: IAuthenticator, IInstaller, IPlugin
Entry point: apache = certbot_apache.entrypoint:ENTRYPOINT

* nginx
Description: Nginx Web Server plugin
Interfaces: IAuthenticator, IInstaller, IPlugin
Entry point: nginx = certbot_nginx.configurator:NginxConfigurator

* standalone
Description: Spin up a temporary webserver
Interfaces: IAuthenticator, IPlugin
Entry point: standalone = certbot.plugins.standalone:Authenticator

* webroot
Description: Place files in webroot directory
Interfaces: IAuthenticator, IPlugin
Entry point: webroot = certbot.plugins.webroot:Authenticator
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Attempts to run certbot-auto using the plugin fail as follows:

    $ sudo ./certbot-auto certonly --dns-cloudflare --dns-cloudflare-credentials ~/.secrets/certbot/cloudflare.ini --dns-cloudflare-propagation-seconds 60 -d my-domain.com
usage:
  certbot-auto [SUBCOMMAND] [options] [-d DOMAIN] [-d DOMAIN] ...

Certbot can obtain and install HTTPS/TLS/SSL certificates.  By default,
it will attempt to use a webserver both for obtaining and installing the
certificate.
certbot: error: unrecognized arguments: --dns-cloudflare-credentials /home/ec2-user/.secrets/certbot/cloudflare.ini --dns-cloudflare-propagation-seconds 60

Can anyone advise? Thanks

Answer 1

I had the same issue trying to install cerbot cloudflare plugin on Amazon Linux. I tried a few different things but the following worked using pip i.e.

sudo yum install -y python-pip
pip install --upgrade pip
pip install certbot-dns-cloudflare

Answer 2

For me certbot was installed in two locations /usr/local/bin/certbot which worked and the default /usr/bin/certbot which couldn't find the newly installed plugins.

I was using which certbot, certbot plugins, and /usr/local/bin/certbot plugins to debug this.

Hope this helps someone.