Thread Info
  • State Not Answered
  • Locked Locked
  • Replies 8 replies
  • Subscribers 3 subscribers
  • Views 2442 views
  • Users 0 members are here
Options
Suggested
This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

UTM SG230/v9.705-3 Let's Encrypt fails

Adrian Ward

Hi all,

Sorry for the long post, but this is tricky to describe to ensure clarity and completeness.

I turned on Let's Encrypt Certificate support on my SG230 and then created a new Let's Encrypt certificate request.  The request from the SG230 fails with the message ...

"An error occurred while communicating with the Let’s Encrypt server. Automatic renewals will be tried again during the next renewal attempt. Manual renewal can be attempted again at any time."

I'm running a number of websites on the server (around 10) and so use the Virtual Webserver functionality of the SG230 and direct requests to the appropriate website accordingly.  I have "Pass Host Header" checked in all cases.  The webserver is IIS.   Prior to activating Let's Encrypt support on the SG230, everything worked just fine (and in fact still does, other than the Let's Encrypt requests originating from the SG230 failing).

Over on IIS, I installed and ran "win-acme" (win-acme) and it seemed to work perfectly.  It created a new certificate request, validated and installed on IIS.  In fact, my sites now have SSL up and running using the certs created and installed by win-acme.

So what the problem you ask?   The error messages on the SG230 when it is trying to create the certs.

Why am I trying to get the SG230 to create the certs as well you ask?  That's because I need to create a Virtual Webserver of the type HTTPS (of course) which requires a certificate which includes the domain in question.  Even though the Let's Encrypt validation fails on the SG230, the Virtual Webserver is created and traffic passes to the correct website on 443.  Sort of a Catch-22 maybe?

I could leave it this way, but having the SG230 poll Let's Encrypt every 24 hours and failing is not really best practice.

Following is the log from the SG230 with domain name and IP changed from the real name and IP.  The answer is probably in the first 4 lines, but I don't know what exit code 256 represents, and I suspect the reference to "/.well-known/" could be because the SG230 is unable to create/write to that directory.

Thanks for any suggestions.

=====================================

2021:05:13-08:03:02 remote letsencrypt[17889]: I Renew certificate: handling CSR REF_CaCsrLetsEncryWwwau for domain set [www.example.com]
2021:05:13-08:03:02 remote letsencrypt[17889]: I Renew certificate: running command: /var/storage/chroot-reverseproxy/usr/dehydrated/bin/dehydrated -x -f /var/storage/chroot-reverseproxy/usr/dehydrated/conf/config -c --accept-terms --domain www.example.com
2021:05:13-08:03:16 remote letsencrypt[17889]: I Renew certificate: command completed with exit code 256
2021:05:13-08:03:16 remote letsencrypt[17889]: E Renew certificate: COMMAND_FAILED: ERROR: Challenge is invalid! (returned: invalid) (result: {
2021:05:13-08:03:16 remote letsencrypt[17889]: E Renew certificate: COMMAND_FAILED: "type": "http-01",
2021:05:13-08:03:16 remote letsencrypt[17889]: E Renew certificate: COMMAND_FAILED: "status": "invalid",
2021:05:13-08:03:16 remote letsencrypt[17889]: E Renew certificate: COMMAND_FAILED: "error": {
2021:05:13-08:03:16 remote letsencrypt[17889]: E Renew certificate: COMMAND_FAILED: "type": "urn:ietf:params:acme:error:unauthorized",
2021:05:13-08:03:16 remote letsencrypt[17889]: E Renew certificate: COMMAND_FAILED: "detail": "Invalid response from www.example.com/.../PKfLUYv_UI3mNo2ywxnJ6hcfEqS2yVOcAlsIQG-vM6A [123.123.123.123]: \"\u003c!DOCTYPE html PUBLIC \\\"-//W3C//DTD XHTML 1.0 Strict//EN\\\" \\\"">www.w3.org/.../xhtml1-strict.dtd\\\"\u003e\\r\\n\u003chtml xmlns=\\\"http\"",
2021:05:13-08:03:16 remote letsencrypt[17889]: E Renew certificate: COMMAND_FAILED: "status": 403
2021:05:13-08:03:16 remote letsencrypt[17889]: E Renew certificate: COMMAND_FAILED: },
2021:05:13-08:03:16 remote letsencrypt[17889]: E Renew certificate: COMMAND_FAILED: "url": "">acme-v02.api.letsencrypt.org/.../61ZNcw",
2021:05:13-08:03:16 remote letsencrypt[17889]: E Renew certificate: COMMAND_FAILED: "token": "PKfLUYv_UI3mNo2ywxnJ6hcfEqS2yVOcAlsIQG-vM6A",
2021:05:13-08:03:16 remote letsencrypt[17889]: E Renew certificate: COMMAND_FAILED: "validationRecord": [
2021:05:13-08:03:16 remote letsencrypt[17889]: E Renew certificate: COMMAND_FAILED: {
2021:05:13-08:03:16 remote letsencrypt[17889]: E Renew certificate: COMMAND_FAILED: "url": "">www.example.com/.../PKfLUYv_UI3mNo2ywxnJ6hcfEqS2yVOcAlsIQG-vM6A",
2021:05:13-08:03:16 remote letsencrypt[17889]: E Renew certificate: COMMAND_FAILED: "hostname": "www.example.com",
2021:05:13-08:03:16 remote letsencrypt[17889]: E Renew certificate: COMMAND_FAILED: "port": "80",
2021:05:13-08:03:16 remote letsencrypt[17889]: E Renew certificate: COMMAND_FAILED: "addressesResolved": [
2021:05:13-08:03:16 remote letsencrypt[17889]: E Renew certificate: COMMAND_FAILED: "123.123.123.123"
2021:05:13-08:03:16 remote letsencrypt[17889]: E Renew certificate: COMMAND_FAILED: ],
2021:05:13-08:03:16 remote letsencrypt[17889]: E Renew certificate: COMMAND_FAILED: "addressUsed": "123.123.123.123"
2021:05:13-08:03:16 remote letsencrypt[17889]: E Renew certificate: COMMAND_FAILED: },
2021:05:13-08:03:16 remote letsencrypt[17889]: E Renew certificate: COMMAND_FAILED: {
2021:05:13-08:03:16 remote letsencrypt[17889]: E Renew certificate: COMMAND_FAILED: "url": "">www.example.com/.../PKfLUYv_UI3mNo2ywxnJ6hcfEqS2yVOcAlsIQG-vM6A",
2021:05:13-08:03:16 remote letsencrypt[17889]: E Renew certificate: COMMAND_FAILED: "hostname": "www.example.com",
2021:05:13-08:03:16 remote letsencrypt[17889]: E Renew certificate: COMMAND_FAILED: "port": "443",
2021:05:13-08:03:16 remote letsencrypt[17889]: E Renew certificate: COMMAND_FAILED: "addressesResolved": [
2021:05:13-08:03:16 remote letsencrypt[17889]: E Renew certificate: COMMAND_FAILED: "123.123.123.123"
2021:05:13-08:03:16 remote letsencrypt[17889]: E Renew certificate: COMMAND_FAILED: ],
2021:05:13-08:03:16 remote letsencrypt[17889]: E Renew certificate: COMMAND_FAILED: "addressUsed": "123.123.123.123"
2021:05:13-08:03:16 remote letsencrypt[17889]: E Renew certificate: COMMAND_FAILED: }
2021:05:13-08:03:16 remote letsencrypt[17889]: E Renew certificate: COMMAND_FAILED: ],
2021:05:13-08:03:16 remote letsencrypt[17889]: E Renew certificate: COMMAND_FAILED: "validated": "2021-05-12T22:03:13Z"
2021:05:13-08:03:16 remote letsencrypt[17889]: E Renew certificate: COMMAND_FAILED: })
2021:05:13-08:03:16 remote letsencrypt[17889]: I Renew certificate: sending notification WARN-603
2021:05:13-08:03:16 remote letsencrypt[17889]: [WARN-603] Let's Encrypt certificate renewal failed accessing Let's Encrypt service
2021:05:13-08:03:16 remote letsencrypt[17889]: I Renew certificate: execution completed (CSRs renewed: 0, failed: 1)

 



This thread was automatically locked due to age.
Parents
  • FormerMember
    0 FormerMember

    Hi ,

    Thanks for reaching out to the Sophos Community! 

    What is the current firmware version on your firewall? Did you configure country blocking or DNAT rules on the external Interface on your firewall; both could cause this issue.

    Thanks,

  • 0 in reply to FormerMember

    Hi Harsh,

    Thanks for your input.

    The current version of the firmware on the SG230 is v9.705-3.

    Country Blocking is not enabled.

    I have around 50 DNAT rules in place, none of which have been altered in recent times.  That is to say, there were no apparent issues prior to enabling Let's Encrypt.  If I focus on the DNAT rules for the moment, what am I looking for that may hinder Let's Encrypt?

    Thanks.

  • FormerMember
    0 FormerMember in reply to Adrian Ward

    Hi ,

    Thank you for the update.

    Check if there’s a DNAT rule with service as ANY. or with service HTTP/HTTPS(port 80/443), try to turn off these rules, and try to create an LE certificate. 

    Thanks,

  • 0 in reply to FormerMember

    I have no DNAT rules with a service of ANY and around 8 DNAT rules with a service of either HTTP (port 80) or HTTP (port 443).  I'm using WAF (Web Application Firewall) for around 10 websites and I have other services on port 443.  If I turn off all the DNAT rules for these services, obviously such things as websites, vpn and mail will stop.

    Are you saying temporarily disable all port 80 and port 443 routing and then try LE until the cert is created and then re-enable the routing (DNAT rules)?  If this is correct, are you saying you want the SG230 to respond to LE, and only the SG230?  What of the error message in the LE log where it is trying to write to a folder on IIS (i.e.  http//www.example.com/.well-known/acme-challenge/)

    Thanks.

  • 0 in reply to Adrian Ward

    So I might have resolved the issue of using the Sophos UTM for Let's Encrypt certificates.

    The solution - Don't use Let's Encrypt on the UTM.  Let IIS look after things.

    On the UTM I deleted the three LE certificate requests in "Webserver Protection\Certificate Management", which deleted the three HTTPS Virtual WebServers in "Webserver Protection\Web Application Firewall" (aka WAF) that I created for the three new websites.

    I then removed LE support by unchecking "Allow Let's Encrypt certificate" in "Webserver Protection\Certificate Management\Advanced".   So ... Let's Encrypt now purged from the SG230 UTM.

    This left the three original HTTPS Virtual WebServers in WAF.  These webservers already have a commercial certificate (not LE).  The Virtual Webservers all have "Pass Host Header" checked and all point to the same Real WebServer, i.e. my IIS server.

    Over on the IIS machine I installed and ran win-acme from https://www.win-acme.com/  This installed and configured the LE certs for the three new websites automatically with little intervention.  Magic.

    All three new websites (domains) are now using SSL and indicate the cert is from LE and valid.

    I don't fully understand why this is working other than the Virtual WebServer in WAF (for the original domains - with the commercial cert installed) is letting/directing requests for the three new domains to pass thru to IIS whereupon IIS is managing the LE certs.

    More than happy for a better explanation, or a correction if I have this wrong, but what can I say - it works.

    To extrapolate ... If you don't have a commercial cert, maybe (?) create a self-signed cert in the UTM to enable the creation of a "dummy" Virtual Webserver entry in WAF, which then points to your real WebServer over on IIS.  Just a thought.

    Hope this might help someone else struggling with a UTM and LE.

  • 0 in reply to Adrian Ward

    Hi Adrian,

    I had the same error from my home lab.  It turned out that my ISP blocks inbound HTTP traffic to residential accounts, so the access by the LetsEncrypt servers never got to the UTM.  My GUESS would be that you have a DNAT for HTTP traffic, so it never "reached" the UTM.

    Cheers - Bob

     
    Sophos UTM Community Moderator
    Sophos Certified Architect - UTM
    Sophos Certified Engineer - XG
    Gold Solution Partner since 2005
    MediaSoft, Inc. USA
Reply
  • 0 in reply to Adrian Ward

    Hi Adrian,

    I had the same error from my home lab.  It turned out that my ISP blocks inbound HTTP traffic to residential accounts, so the access by the LetsEncrypt servers never got to the UTM.  My GUESS would be that you have a DNAT for HTTP traffic, so it never "reached" the UTM.

    Cheers - Bob

     
    Sophos UTM Community Moderator
    Sophos Certified Architect - UTM
    Sophos Certified Engineer - XG
    Gold Solution Partner since 2005
    MediaSoft, Inc. USA
Children
  • 0 in reply to BAlfson

    Thanks Bob for your thoughts on this issue.

    The site is a commercial site, so no ISP issues with HTTP or HTTPS requests sent to our IP/domain.

    Currently access by the Let's Encrypt servers would seem to be fine as IIS seems to manage Let's Encrypt OK.  The issue is the UTM9 (SG230) just doesn't want to play ball with Let's Encrypt.

    The DNATs that are setup send inbound traffic (both port 80 and 443) to the webserver (IIS).  Maybe I need to temporarily disable the Virtual Webservers and Real Webservers in Web Application Firewall (WAF) so that the Let's Encrypt requests don't get sent to IIS at all and just end at the UTM (??) - at least until the certificate is created.

    While I managed to work-around the UTM9 (SG230) Let's Encrypt problem by giving up on the UTM9 and just letting IIS look after things, it is kind of annoying that the UTM9 box is not really doing what it is supposed to be able to do.

    Cheers - Adrian

  • 0 in reply to Adrian Ward

    I was fairly certain that your problem was that DNAT, Adrian.  Above the DNAT, make a NoNAT rule for port 80/443 traffic from the LE servers.  Sorry, I don't know what IPs or FQDNs should be used.

    Cheers - Bob

     
    Sophos UTM Community Moderator
    Sophos Certified Architect - UTM
    Sophos Certified Engineer - XG
    Gold Solution Partner since 2005
    MediaSoft, Inc. USA
Unfiltered HTML