Let's Encrypt renewal problem

Hi

Yes, a few days ago I had a failure to auto-renew.

When I first tried obtaining a LE certificate a few times (a couple of months back) I failed miserably, then I spotted a post suggesting that country blocking could prevent the process (which makes sense as I had USA set to block 'From'; again, oops) so after letting the USA back through, I requested my certificate again and this time, it worked.

Folloing the successful installation of the certificate, I then set the USA back to 'From' and instead created a country blocking exception for acme-v01.api.letsencrypt.org and once again, I hit the 'Renew' button in the Certificate Management section, and that also worked; my certificate was successfully updated. I waited a while and tried again, and once again the renewal process worked, so I was pretty confident that I'd found the solution.*

That all said, another poster then tried that same trick and for some odd reason, they were unsuccessful.

Move forward to last week (and with me now running 9.603-1) and I received an email from UTM stating that the automatic certificate renewal process had failed, so I again tried the manual renew process and that also failed. I wondered if the LE domain had been changed, but looking at the logs showed that it was still acme-v01.api.letsencrypt.org (implying that it hadn't changed) so this time I had to let the USA back through country blocking (and then after hitting the manual refresh button, this time it worked) so in my case, it certainly looks like something associated with the 9.603-1 update is perhaps now preventing my country blocking exception for acme-v01.api.letsencrypt.org from working.

Anyhow, it's no big deal for me as I can simply let the USA back through after receiving my next failure e-mail (assuming it does fail, that is) and then manually hit the 'refresh' button.

Bri

*Incidentally, after re-testing that country blocking exception yet one more time - just to prove it was 100% reliable - it failed to renew the certificate. I looked at the LE logs and at the LE site, discovering that you can only apply 5 times (in any 5 day rolling period) then you get biffed off the LE server for 5 days; oops! Of course, it wasn't an issue as my existing certificate still had 90 days of life left in it.

Hi

Yes, a few days ago I had a failure to auto-renew.

When I first tried obtaining a LE certificate a few times (a couple of months back) I failed miserably, then I spotted a post suggesting that country blocking could prevent the process (which makes sense as I had USA set to block 'From'; again, oops) so after letting the USA back through, I requested my certificate again and this time, it worked.

Folloing the successful installation of the certificate, I then set the USA back to 'From' and instead created a country blocking exception for acme-v01.api.letsencrypt.org and once again, I hit the 'Renew' button in the Certificate Management section, and that also worked; my certificate was successfully updated. I waited a while and tried again, and once again the renewal process worked, so I was pretty confident that I'd found the solution.*

That all said, another poster then tried that same trick and for some odd reason, they were unsuccessful.

Move forward to last week (and with me now running 9.603-1) and I received an email from UTM stating that the automatic certificate renewal process had failed, so I again tried the manual renew process and that also failed. I wondered if the LE domain had been changed, but looking at the logs showed that it was still acme-v01.api.letsencrypt.org (implying that it hadn't changed) so this time I had to let the USA back through country blocking (and then after hitting the manual refresh button, this time it worked) so in my case, it certainly looks like something associated with the 9.603-1 update is perhaps now preventing my country blocking exception for acme-v01.api.letsencrypt.org from working.

Anyhow, it's no big deal for me as I can simply let the USA back through after receiving my next failure e-mail (assuming it does fail, that is) and then manually hit the 'refresh' button.

Bri

*Incidentally, after re-testing that country blocking exception yet one more time - just to prove it was 100% reliable - it failed to renew the certificate. I looked at the LE logs and at the LE site, discovering that you can only apply 5 times (in any 5 day rolling period) then you get biffed off the LE server for 5 days; oops! Of course, it wasn't an issue as my existing certificate still had 90 days of life left in it.

Hi Briain,

Thanks for replying.  I live in the US, so USA blocking doesn't apply.  Although acme-v01.x.x.x domain shows in the log, I went ahead and make an exception and do a manual renewal and my LE still failed.

BTW, I'm also aware of the 5 days limitation on the manual renewal

Ah, well your location rather rules out my country blocking issue! :-)

Anyhow, with me living in Scotland, with the USA permitted through country blocking and after a manual renew, it did actually work, so that implies 9.603-1 is capable of successfully refreshing an LE certificate.

Bri

Hi

I've just had a looked at my archive logs to see what was shown for the auto-renew failure:

2019:06:23-02:38:01 hadrian letsencrypt[8147]: I Check renewal: renew REF_CaCsrLetsencryp (domains: --------.ddns.net): certificate valid until Jul 22 12:44:18 2019 GMT (less than 30 days)
2019:06:23-02:39:03 hadrian letsencrypt[8492]: I Renew certificate: handling CSR REF_CaCsrLetsencryp for domain set [--------.ddns.net]
2019:06:23-02:39:03 hadrian letsencrypt[8492]: I Renew certificate: running command: /var/storage/chroot-reverseproxy/usr/dehydrated/bin/dehydrated -x -f /var/storage/chroot-reverseproxy/usr/dehydrated/conf/config -c --accept-terms --domain --------.ddns.net
2019:06:23-02:39:24 hadrian letsencrypt[8492]: I Renew certificate: command completed with exit code 256
2019:06:23-02:39:24 hadrian letsencrypt[8492]: E Renew certificate: COMMAND_FAILED: ERROR: Challenge is invalid! (returned: invalid) (result: {
2019:06:23-02:39:24 hadrian letsencrypt[8492]: E Renew certificate: COMMAND_FAILED:   "type": "http-01",
2019:06:23-02:39:24 hadrian letsencrypt[8492]: E Renew certificate: COradsMMAND_FAILED:   "status": "invalid",
2019:06:23-02:39:24 hadrian letsencrypt[8492]: E Renew certificate: COMMAND_FAILED:   "error": {
2019:06:23-02:39:24 hadrian letsencrypt[8492]: E Renew certificate: COMMAND_FAILED:     "type": "urn:acme:error:connection",
2019:06:23-02:39:24 hadrian letsencrypt[8492]: E Renew certificate: COMMAND_FAILED:     "detail": "Fetching --------.ddns.net/.../ly2GTBx4zNk_rhT9_5VzdXMo1Cv7cL3OzJfIfnMPWJY: Timeout during connect (likely firewall problem)",
2019:06:23-02:39:24 hadrian letsencrypt[8492]: E Renew certificate: COMMAND_FAILED:     "status": 400
2019:06:23-02:39:24 hadrian letsencrypt[8492]: E Renew certificate: COMMAND_FAILED:   },
2019:06:23-02:39:24 hadrian letsencrypt[8492]: E Renew certificate: COMMAND_FAILED:   "uri": "acme-v01.api.letsencrypt.org/.../17402800189",
2019:06:23-02:39:24 hadrian letsencrypt[8492]: E Renew certificate: COMMAND_FAILED:   "token": "ly2GTBx4zNk_rhT9_5VzdXMo1Cv7cL3OzJfIfnMPWJY",
2019:06:23-02:39:24 hadrian letsencrypt[8492]: E Renew certificate: COMMAND_FAILED:   "validationRecord": [
2019:06:23-02:39:24 hadrian letsencrypt[8492]: E Renew certificate: COMMAND_FAILED:     {
2019:06:23-02:39:24 hadrian letsencrypt[8492]: E Renew certificate: COMMAND_FAILED:       "url": "--------.ddns.net/.../ly2GTBx4zNk_rhT9_5VzdXMo1Cv7cL3OzJfIfnMPWJY",
2019:06:23-02:39:24 hadrian letsencrypt[8492]: E Renew certificate: COMMAND_FAILED:       "hostname": "--------.ddns.net",
2019:06:23-02:39:24 hadrian letsencrypt[8492]: E Renew certificate: COMMAND_FAILED:       "port": "80",
2019:06:23-02:39:24 hadrian letsencrypt[8492]: E Renew certificate: COMMAND_FAILED:       "addressesResolved": [
2019:06:23-02:39:24 hadrian letsencrypt[8492]: E Renew certificate: COMMAND_FAILED:         "---.---.---.132"
2019:06:23-02:39:24 hadrian letsencrypt[8492]: E Renew certificate: COMMAND_FAILED:       ],
2019:06:23-02:39:24 hadrian letsencrypt[8492]: E Renew certificate: COMMAND_FAILED:       "addressUsed": "---.---.---.132"
2019:06:23-02:39:24 hadrian letsencrypt[8492]: E Renew certificate: COMMAND_FAILED:     }
2019:06:23-02:39:24 hadrian letsencrypt[8492]: E Renew certificate: COMMAND_FAILED:   ]
2019:06:23-02:39:24 hadrian letsencrypt[8492]: E Renew certificate: COMMAND_FAILED: })
2019:06:23-02:39:25 hadrian letsencrypt[8492]: I Renew certificate: sending notification WARN-603
2019:06:23-02:39:25 hadrian letsencrypt[8492]: [WARN-603] Let's Encrypt certificate renewal failed accessing Let's Encrypt service
2019:06:23-02:39:25 hadrian letsencrypt[8492]: I Renew certificate: execution completed (CSRs renewed: 0, failed: 1)

So your log shows similar to the failure to the one that I'd had (before then letting the USA through) and thus that the response from LE is being blocked somewhere.

Sorry to ask a such a silly question (though this one did initially catch me out) but are you using a double NAT scheme (e.g. an ISP router in front of UTM) and thus could it just be that for some odd reason, port 80 is no longer open [to the UTM WAN address] in the ISP router?

Bri

Hi Briain,

I'm not using double-NAT.  Just your typical setup modem in front of the UTM.

Hi

Yes, sorry but it was just rather a wild 'stab in the dark' and I realise that most USA ISP's issue modems (whereas in the UK, it's almost always routers; the only way we can avoid double NAT is to use a Draytek V130).

I'm struggling to think what else could be blocking it, but over here, it's time to make dinner, so I'll ponder it all further over a glass of red wine (though at the moment, I cannot think of anything else that caught me out when initially doing all this).

Bri

PS Enjoy the 4th July celebrations over there; I hope one already has one's has a BBQ lit and that the beers are well chilled (we don't get many chances to do BBQs in Scotland; typically, all the food - and the BBQ fuel - almost immediately get blown away in the >70 MPH westerly 'breezes')! :-)

Hi Briain,

My issue is now resolved.  My cert was using an old Interface.  Deleted my cert and re-create and point to the new Interface and it works.  Thanks for all your feedback and hope you have a pleasant dinner.  Looking forward to the fireworks.