Thread Info
  • State Verified Answer
  • +2 person also asked this people also asked this
  • Replies 2 replies
  • Subscribers 6 subscribers
  • Views 22992 views
  • Users 0 members are here
Options
Suggested

Let's encrypt: Error challenge is invalid

apijnappels

I'm trying to get a LE certificate for 2 domainnames that both point to the same IP-address (which is the same UTM on it's external interface).

However cert doesn't get generated, see log for errors (I have partly obfuscated the domainnames and my IP-address):

2018:09:26-14:54:02 firewall letsencrypt[9407]: I Renew certificate: handling CSR REF_CaCsrLetsEncryUtm for domain set [firewall.****s.eu,utm.****s.eu]
2018:09:26-14:54:02 firewall letsencrypt[9407]: I Renew certificate: running command: /var/storage/chroot-reverseproxy/usr/dehydrated/bin/dehydrated -x -f /var/storage/chroot-reverseproxy/usr/dehydrated/conf/config -c --accept-terms --domain firewall.****s.eu --domain utm.****s.eu
2018:09:26-14:54:08 firewall letsencrypt[9407]: I Renew certificate: command completed with exit code 256
2018:09:26-14:54:08 firewall letsencrypt[9407]: E Renew certificate: COMMAND_FAILED: ERROR: Challenge is invalid! (returned: invalid) (result: {
2018:09:26-14:54:08 firewall letsencrypt[9407]: E Renew certificate: COMMAND_FAILED: "type": "http-01",
2018:09:26-14:54:08 firewall letsencrypt[9407]: E Renew certificate: COMMAND_FAILED: "status": "invalid",
2018:09:26-14:54:08 firewall letsencrypt[9407]: E Renew certificate: COMMAND_FAILED: "error": {
2018:09:26-14:54:08 firewall letsencrypt[9407]: E Renew certificate: COMMAND_FAILED: "type": "urn:acme:error:unauthorized",
2018:09:26-14:54:08 firewall letsencrypt[9407]: E Renew certificate: COMMAND_FAILED: "detail": "Invalid response from firewall.****s.eu/.../TNi4gpGYdTnXumYHVFQIjuEw0f1wweFhNbEfIceKr38: q%!(EXTRA string=\u003c!DOCTYPE html\u003e\n\u003chtml\u003e\n\u003chead\u003e\n\u003cmeta charset=\"utf-8\"\u003e\n\u003cstyle\u003ebody{font-family:Arial,Helvetica,sans-serif;font-size:12px;text-alig)",
2018:09:26-14:54:08 firewall letsencrypt[9407]: E Renew certificate: COMMAND_FAILED: "status": 403
2018:09:26-14:54:08 firewall letsencrypt[9407]: E Renew certificate: COMMAND_FAILED: },
2018:09:26-14:54:08 firewall letsencrypt[9407]: E Renew certificate: COMMAND_FAILED: "uri": "acme-v01.api.letsencrypt.org/.../7689019129",
2018:09:26-14:54:08 firewall letsencrypt[9407]: E Renew certificate: COMMAND_FAILED: "token": "TNi4gpGYdTnXumYHVFQIjuEw0f1wweFhNbEfIceKr38",
2018:09:26-14:54:08 firewall letsencrypt[9407]: E Renew certificate: COMMAND_FAILED: "validationRecord": [
2018:09:26-14:54:08 firewall letsencrypt[9407]: E Renew certificate: COMMAND_FAILED: {
2018:09:26-14:54:08 firewall letsencrypt[9407]: E Renew certificate: COMMAND_FAILED: "url": "firewall.****s.eu/.../TNi4gpGYdTnXumYHVFQIjuEw0f1wweFhNbEfIceKr38",
2018:09:26-14:54:08 firewall letsencrypt[9407]: E Renew certificate: COMMAND_FAILED: "hostname": "firewall.****s.eu",
2018:09:26-14:54:08 firewall letsencrypt[9407]: E Renew certificate: COMMAND_FAILED: "port": "80",
2018:09:26-14:54:08 firewall letsencrypt[9407]: E Renew certificate: COMMAND_FAILED: "addressesResolved": [
2018:09:26-14:54:08 firewall letsencrypt[9407]: E Renew certificate: COMMAND_FAILED: "84.*.*.89"
2018:09:26-14:54:08 firewall letsencrypt[9407]: E Renew certificate: COMMAND_FAILED: ],
2018:09:26-14:54:08 firewall letsencrypt[9407]: E Renew certificate: COMMAND_FAILED: "addressUsed": "84.*.*.89"
2018:09:26-14:54:08 firewall letsencrypt[9407]: E Renew certificate: COMMAND_FAILED: }
2018:09:26-14:54:08 firewall letsencrypt[9407]: E Renew certificate: COMMAND_FAILED: ]
2018:09:26-14:54:08 firewall letsencrypt[9407]: E Renew certificate: COMMAND_FAILED: })
2018:09:26-14:54:08 firewall letsencrypt[9407]: I Renew certificate: sending notification WARN-603
2018:09:26-14:54:08 firewall letsencrypt[9407]: [WARN-603] Let's Encrypt certificate renewal failed accessing Let's Encrypt service
2018:09:26-14:54:08 firewall letsencrypt[9407]: I Renew certificate: execution completed (CSRs renewed: 0, failed: 1)

  • 0

    It looks like something's turning the plaintext ACME challenge into a HTML page. See the "EXTRA string" in the log with DOCTYPE and other HTML stuff.

    Just to make sure: There's nothing between WAF and Let's Encrypt, is that correct? Like a load balancer or some other "active" network component that might mess with the responses from WAF?

    Here's how you can help us to figure this out:

    • Post the line in /var/log/reverseproxy.log that belongs to the ACME challenge request (search for "/.well-known/acme-challenge")
    • Disable the certificate renewal process on the command line for the next steps
      touch /tmp/disable_letsencrypt_renew_certificates
    • In WebAdmin click on the "Renew" button next to your failing certificate
    • Attach /var/chroot-reverseproxy/usr/apache/conf/reverseproxy.conf to your post
    • Of course anonymize any sensitive information in there
    • If this file is too big or you feel uncomfortable posting it here, we will find another way to get the information in it we need
    • Create a small text file in the ACME challenge location
      echo "test" > /var/chroot-reverseproxy/var/letsencrypt/acme-challenge/test
    • It would be ideal if you could request that file with a command line tool like curl including the HTTP headers
      curl -i http://<your domain>/.well-known/acme-challenge/test
    • Otherwise use a browser and check if it's returned as a plaintext file or a HTML page
    • If you used curl, post it's response (including the HTTP headers) here
    • When done you can remove the ACME challenge test file
      rm /var/chroot-reverseproxy/var/letsencrypt/acme-challenge/test
    • Re-enable the certificate renewal process by removing the file that we created at the beginning. It will execute the certificate renewal a little bit later and you will probably receive another failure notification.
      rm /tmp/disable_letsencrypt_renew_certificates

    That's a lot of steps, so thanks for your help!

  • +3 in reply to ewadie

    Found the reason... There was still an active DNAT directly connecting http and https to an inside webserver (also doing Let's Encrypt).

    When I disabled the DNAT rule the renew worked as it should. Should have seen that coming, by bad.

    Managing several Sophos firewalls both at work and at some home locations, dedicated to continuously improve IT-security and feeling well helping others with their IT-security challenges.

Unfiltered HTML