This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

9.605-1 Let's Encrypt renewals fail

Hello,

Struggling to get this to work. Initial requests work fine, but renewals always fail.

Letsencrypt log says:

2019:09:17-15:27:02 firewall-1 letsencrypt[12156]: I Renew certificate: handling CSR REF_CaCsrSupportwer for domain set [support.werkplanner.info]
2019:09:17-15:27:02 firewall-1 letsencrypt[12156]: I Renew certificate: running command: /var/storage/chroot-reverseproxy/usr/dehydrated/bin/dehydrated -x -f /var/storage/chroot-reverseproxy/usr/dehydrated/conf/config -c --accept-terms --domain support.werkplanner.info
2019:09:17-15:27:10 firewall-1 letsencrypt[12156]: I Renew certificate: command completed with exit code 256
2019:09:17-15:27:10 firewall-1 letsencrypt[12156]: E Renew certificate: COMMAND_FAILED: ERROR: Challenge is invalid! (returned: invalid) (result: {
2019:09:17-15:27:10 firewall-1 letsencrypt[12156]: E Renew certificate: COMMAND_FAILED: "type": "http-01",
2019:09:17-15:27:10 firewall-1 letsencrypt[12156]: E Renew certificate: COMMAND_FAILED: "status": "invalid",
2019:09:17-15:27:10 firewall-1 letsencrypt[12156]: E Renew certificate: COMMAND_FAILED: "error": {
2019:09:17-15:27:10 firewall-1 letsencrypt[12156]: E Renew certificate: COMMAND_FAILED: "type": "urn:acme:error:unauthorized",
2019:09:17-15:27:10 firewall-1 letsencrypt[12156]: E Renew certificate: COMMAND_FAILED: "detail": "Invalid response from support.werkplanner.info:443/.../vgDi2_L0jHfUHY7HgbhQol6XIPUBhCI05zuM5gzhxF4 [2001:1b40:4000:5::219:77]: \"\u003c!DOCTYPE html\u003e\\n\u003chtml\u003e\\n\u003chead\u003e\\n \u003cmeta charset=\\\"utf-8\\\" /\u003e\\n \u003ctitle\u003eRedmine 404 error\u003c/title\u003e\\n \u003cstyle\u003e\\n body {font-family: \\\"Tr\"",
2019:09:17-15:27:10 firewall-1 letsencrypt[12156]: E Renew certificate: COMMAND_FAILED: "status": 403
2019:09:17-15:27:10 firewall-1 letsencrypt[12156]: E Renew certificate: COMMAND_FAILED: },
2019:09:17-15:27:10 firewall-1 letsencrypt[12156]: E Renew certificate: COMMAND_FAILED: "uri": "acme-v01.api.letsencrypt.org/.../......",
2019:09:17-15:27:10 firewall-1 letsencrypt[12156]: E Renew certificate: COMMAND_FAILED: "token": "........",
2019:09:17-15:27:10 firewall-1 letsencrypt[12156]: E Renew certificate: COMMAND_FAILED: "validationRecord": [
2019:09:17-15:27:10 firewall-1 letsencrypt[12156]: E Renew certificate: COMMAND_FAILED: {
2019:09:17-15:27:10 firewall-1 letsencrypt[12156]: E Renew certificate: COMMAND_FAILED: "url": "support.werkplanner.info/.../vgDi2_L0jHfUHY7HgbhQol6XIPUBhCI05zuM5gzhxF4",
2019:09:17-15:27:10 firewall-1 letsencrypt[12156]: E Renew certificate: COMMAND_FAILED: "hostname": "support.werkplanner.info",
2019:09:17-15:27:10 firewall-1 letsencrypt[12156]: E Renew certificate: COMMAND_FAILED: "port": "80",
2019:09:17-15:27:10 firewall-1 letsencrypt[12156]: E Renew certificate: COMMAND_FAILED: "addressesResolved": [
2019:09:17-15:27:10 firewall-1 letsencrypt[12156]: E Renew certificate: COMMAND_FAILED: "78.129.219.77",
2019:09:17-15:27:10 firewall-1 letsencrypt[12156]: E Renew certificate: COMMAND_FAILED: "2001:1b40:4000:5::219:77"
2019:09:17-15:27:10 firewall-1 letsencrypt[12156]: E Renew certificate: COMMAND_FAILED: ],
2019:09:17-15:27:10 firewall-1 letsencrypt[12156]: E Renew certificate: COMMAND_FAILED: "addressUsed": "2001:1b40:4000:5::219:77"
2019:09:17-15:27:10 firewall-1 letsencrypt[12156]: E Renew certificate: COMMAND_FAILED: },
2019:09:17-15:27:10 firewall-1 letsencrypt[12156]: E Renew certificate: COMMAND_FAILED: {
2019:09:17-15:27:10 firewall-1 letsencrypt[12156]: E Renew certificate: COMMAND_FAILED: "url": "support.werkplanner.info:443/.../vgDi2_L0jHfUHY7HgbhQol6XIPUBhCI05zuM5gzhxF4",
2019:09:17-15:27:10 firewall-1 letsencrypt[12156]: E Renew certificate: COMMAND_FAILED: "hostname": "support.werkplanner.info",
2019:09:17-15:27:10 firewall-1 letsencrypt[12156]: E Renew certificate: COMMAND_FAILED: "port": "443",
2019:09:17-15:27:10 firewall-1 letsencrypt[12156]: E Renew certificate: COMMAND_FAILED: "addressesResolved": [
2019:09:17-15:27:10 firewall-1 letsencrypt[12156]: E Renew certificate: COMMAND_FAILED: "78.129.219.77",
2019:09:17-15:27:10 firewall-1 letsencrypt[12156]: E Renew certificate: COMMAND_FAILED: "2001:1b40:4000:5::219:77"
2019:09:17-15:27:10 firewall-1 letsencrypt[12156]: E Renew certificate: COMMAND_FAILED: ],
2019:09:17-15:27:10 firewall-1 letsencrypt[12156]: E Renew certificate: COMMAND_FAILED: "addressUsed": "2001:1b40:4000:5::219:77"
2019:09:17-15:27:10 firewall-1 letsencrypt[12156]: E Renew certificate: COMMAND_FAILED: }
2019:09:17-15:27:10 firewall-1 letsencrypt[12156]: E Renew certificate: COMMAND_FAILED: ]
2019:09:17-15:27:10 firewall-1 letsencrypt[12156]: E Renew certificate: COMMAND_FAILED: })
2019:09:17-15:27:10 firewall-1 letsencrypt[12156]: I Renew certificate: sending notification WARN-603
2019:09:17-15:27:10 firewall-1 letsencrypt[12156]: [WARN-603] Let's Encrypt certificate renewal failed accessing Let's Encrypt service
2019:09:17-15:27:10 firewall-1 letsencrypt[12156]: I Renew certificate: execution completed (CSRs renewed: 0, failed: 1)

Issue seems to be that the UTM doesn't capture the validation request, as the WAF has passed it through to the webserver, which returns a 404 as seen in the detailed response.

The log of the webserver confirms this:

2600:1f16:269:da00:4ec6:1cf7:34d5:6263 - - [17/Sep/2019:15:27:09 +0000] "GET /.well-known/acme-challenge/......... HTTP/1.0" 404 459 "support.werkplanner.info/.../......." "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)" [0s,7627us]
2600:3000:2710:200::1e - - [17/Sep/2019:15:27:09 +0000] "GET /.well-known/acme-challenge/......... HTTP/1.0" 404 459 "support.werkplanner.info/.../......." "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)" [0s,3837us]

I double checked the certificate definition, it does define the same interface as the WAF definition (if not the initial request would fail as well I would think?).

This is the 4th time this happens, until now I've just deleted the cert definition and recreated it, but it is supposed to happen automatically.

However, I have no clue where to start searching. Anyone with the golden tip?



This thread was automatically locked due to age.
  • rabla said:

    With 9.702 there is also an impact of Contry Blocking. It seems to work with HTTPS redirection enabled, but not with Country Blocking enabled.

    Would you mind to start a separate discussion thread for that? It almost certainly has nothing to do with the Let's Encrypt issue from this thread.

    mle