Installing a LetsEncrypt certificate on an HPE iLO 5

Once again, I’ve spent far too long trying to work out how the heck to get a LetsEncrypt X.509 certificate on to the HPE Integrated Lights-Out 5 board.

To save me some time in three months, and to save you some time since you’re already here, the instructions are really straightforward. Be aware that this method uses a DNS-based challenge which may be tricky for you to do unless you can automate DNS updates.

First, from the iLO web interface, select Security and SSL Certificate, and click Customize Certificiate.

Next, Click Generate CSR and go back to the page in a few minutes when the certificate signing request (CSR) has been generated.

Copy the CSR in to a file on a machine with the ‘certbot’ client installed and run the following command:

certbot certonly --csr request.csr --preferred-challenges dns

When the certificate has been issued, select Import Certificate and paste in the entire PEM-formatted file.

And it’s really as straightforward as that.

Let’s Encrypt and Zabbix Agents

A week or two ago, some of a number of servers became unreachable via Zabbix. Both pairs of servers (as we run everything in pairs) showed the same problem, and no other host did.

Looking in the Zabbix log showed this rather cryptic error:

SSL_shutdown() with 172.31.16.32 set result code to 6

That’s not very helpful, and several hours of head-scratching went by before we finally stumbled across what was happening.

The X.509 (or SSL) certificate on the target machine is issued by Let’s Encrypt, who are in the process of signing new certificates with a new key. Within Zabbix, we check that the certificate presented by the client when we connect is issued by a specific issuer and since this had changed, the server was refusing to connect.

How did we fix it? Really easily – by going in to the host configuration in Zabbix, and setting the issuer to:

CN=R3,O=Let's Encrypt,C=US

Ridiculously straightforward, and if you hover over the red ‘ZBX’ status box, you’ll see an error saying that the wrong issuer was found on the certificate.

That’s a few hours we’ll never get back, but it’s great to have solved the problem. And as it happens, there was a correlation between the servers affected – two pairs were built at the same time, and the remaining pair was built just about 90 days before the others. Let’s Encrypt certificates have a lifetime of 90 days.

We’re expecting further servers to drop off Zabbix, but at least we know why and we can fix it.