Let's Encrypt certificates

Why a cluster makes this hard

An http-01 challenge works by the certificate authority asking for a file over port 80 at the name you are claiming. Only the node that currently owns that public address can answer. Which node that is changes when you fail over, and nothing about your certbot setup notices.

The usual result is a timer that fires on every box, succeeds on one and fails on the others, and fills somebody's mailbox with failure mail until the real failure is buried in it.

How it is done here

1. At every renewal, the name is looked up against public DNS. Not cached, not assumed, looked up.
2. The answer is compared against what each node reports its own public address to be.
3. The node that matches is the one that runs certbot, because it is the only one the authority can reach.
4. The challenge token is written to every node before validation starts, because the authority picks which address it connects to and you do not get a say.
5. The issued certificate and key are copied to every node. The key stays at mode 0600 and owned by root.

The certbot systemd timer is switched off during installation, on purpose. Renewal is driven from the manager, which knows which node can answer.

Getting your first one

1. Point the name at one of your load balancers in DNS, and wait for that to be true everywhere.
2. Make sure port 80 is open from the internet and the site answers on it.
3. Go to Certificates, press Get one from Let's Encrypt, and type the domain. Add more names if one certificate should cover several.
4. Press Request. It takes about thirty seconds.
5. Open the site, choose the certificate, turn on Listen on https and Force https, then apply.

Renewal

Automatic, thirty days before expiry by default. That leaves a fortnight to notice a failure and still have plenty of time to fix it. You do not need a calendar reminder, and if a renewal does fail, the certificate list says so and it keeps trying, so one bad night does not become an outage a month later.

When http-01 will not work

Two cases need dns-01 instead, where the authority asks you to put a record in your DNS.

• A wildcard certificate covering anything.example.com. It cannot be done any other way.
• A name that is not reachable from the internet at all, such as an internal service that still wants a trusted certificate.

Put your DNS provider credentials in Settings and the manager does it for you. They are stored encrypted and never shown again.

When it does not work

Common questions