Thread Info
  • State Verified Answer
  • Locked Locked
  • Replies 9 replies
  • Subscribers 3 subscribers
  • Views 6549 views
  • Users 0 members are here
Options
Suggested
This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

Certbot (Let's Encrypt) verification not getting through WAF.

jlbrown

I'm trying to get Let's Encrypt working on one of my web servers (which is behind the WAF).

This is what I'm running and receiving on the web server:

$ sudo certbot certonly --webroot -d internal.bordo.com.au -w /myRoot/
Password:
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator webroot, Installer None
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for internal.bordo.com.au
Using the webroot path /myRoot for all unmatched domains.
Waiting for verification...
Challenge failed for domain internal.bordo.com.au
http-01 challenge for internal.bordo.com.au
Cleaning up challenges
Some challenges have failed.

IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: internal.bordo.com.au
   Type:   connection
   Detail: Fetching
   internal.bordo.com.au/.../PftAsVdmkyiDzNJpxfjGkuat-vviRTpiTtZRyuOGPrw:
   Timeout after connect (your server may be slow or overloaded)

   To fix these errors, please make sure that your domain name was
   entered correctly and the DNS A/AAAA record(s) for that domain
   contain(s) the right IP address. Additionally, please check that
   your computer has a publicly routable IP address and that no
   firewalls are preventing the server from communicating with the
   client. If you're using the webroot plugin, you should also verify
   that you are serving files from the webroot path you provided.

The WAF log (logwin.html) shows:

2020:11:17-10:55:57 astaro1-1 httpd: id="0299" srcip="3.128.26.105" localip="139.130.139.174" size="257" user="-" host="3.128.26.105" method="GET" statuscode="301" reason="-" extra="-" exceptions="-" time="74965" url="/.well-known/acme-challenge/V7nRDlBP_91-Td-6ISQuAVMw9BgEai4syOROvNXN8i0" server="internal.bordo.com.au" port="80" query="" referer="-" cookie="-" set-cookie="-" websocket_scheme="-" websocket_protocol="-" websocket_key="-" websocket_version="-" uid="X7MRjXkFxqwNTKr@DgGTfQAAAAk"
2020:11:17-10:55:57 astaro1-1 httpd: id="0299" srcip="34.211.6.84" localip="139.130.139.174" size="257" user="-" host="34.211.6.84" method="GET" statuscode="301" reason="-" extra="-" exceptions="-" time="83948" url="/.well-known/acme-challenge/V7nRDlBP_91-Td-6ISQuAVMw9BgEai4syOROvNXN8i0" server="internal.bordo.com.au" port="80" query="" referer="-" cookie="-" set-cookie="-" websocket_scheme="-" websocket_protocol="-" websocket_key="-" websocket_version="-" uid="X7MRjXkFxqwNTKr@DgGTeQAAAAw"
2020:11:17-10:55:57 astaro1-1 httpd: id="0299" srcip="127.0.0.1" localip="127.0.0.1" size="46875" user="-" host="127.0.0.1" method="GET" statuscode="200" reason="-" extra="-" exceptions="-" time="26763" url="/status" server="localhost:4080" port="80" query="" referer="-" cookie="-" set-cookie="-" websocket_scheme="-" websocket_protocol="-" websocket_key="-" websocket_version="-" uid="X7MRjXkFxqwNTKr@DgGTmAAAACc"
2020:11:17-10:55:57 astaro1-1 httpd: id="0299" srcip="52.28.236.88" localip="139.130.139.174" size="257" user="-" host="52.28.236.88" method="GET" statuscode="301" reason="-" extra="-" exceptions="-" time="82802" url="/.well-known/acme-challenge/V7nRDlBP_91-Td-6ISQuAVMw9BgEai4syOROvNXN8i0" server="internal.bordo.com.au" port="80" query="" referer="-" cookie="-" set-cookie="-" websocket_scheme="-" websocket_protocol="-" websocket_key="-" websocket_version="-" uid="X7MRjXkFxqwNTKr@DgGTgQAAAAQ"
2020:11:17-10:55:57 astaro1-1 httpd: id="0299" srcip="64.78.149.164" localip="139.130.139.174" size="257" user="-" host="64.78.149.164" method="GET" statuscode="301" reason="-" extra="-" exceptions="-" time="103247" url="/.well-known/acme-challenge/V7nRDlBP_91-Td-6ISQuAVMw9BgEai4syOROvNXN8i0" server="internal.bordo.com.au" port="80" query="" referer="-" cookie="-" set-cookie="-" websocket_scheme="-" websocket_protocol="-" websocket_key="-" websocket_version="-" uid="X7MRjXkFxqwNTKr@DgGTdwAAAAE"
Another try:
2020:11:17-11:34:52 astaro1-1 httpd: id="0299" srcip="3.128.26.105" localip="139.130.139.174" size="254" user="-" host="3.128.26.105" method="GET" statuscode="301" reason="-" extra="-" exceptions="-" time="216282" url="/.well-known/acme-challenge/PftAsVdmkyiDzNJpxfjGkuat-vviRTpiTtZRyuOGPrw" server="internal.bordo.com.au" port="80" query="" referer="-" cookie="-" set-cookie="-" websocket_scheme="-" websocket_protocol="-" websocket_key="-" websocket_version="-" uid="X7Maq3BZLmB8Joos1mMIaAAAAGE"
2020:11:17-11:34:52 astaro1-1 httpd: id="0299" srcip="18.196.96.172" localip="139.130.139.174" size="254" user="-" host="18.196.96.172" method="GET" statuscode="301" reason="-" extra="-" exceptions="-" time="8947" url="/.well-known/acme-challenge/PftAsVdmkyiDzNJpxfjGkuat-vviRTpiTtZRyuOGPrw" server="internal.bordo.com.au" port="80" query="" referer="-" cookie="-" set-cookie="-" websocket_scheme="-" websocket_protocol="-" websocket_key="-" websocket_version="-" uid="X7MarHBZLmB8Joos1mMIaQAAAEA"
2020:11:17-11:34:52 astaro1-1 httpd: id="0299" srcip="34.211.6.84" localip="139.130.139.174" size="254" user="-" host="34.211.6.84" method="GET" statuscode="301" reason="-" extra="-" exceptions="-" time="3508" url="/.well-known/acme-challenge/PftAsVdmkyiDzNJpxfjGkuat-vviRTpiTtZRyuOGPrw" server="internal.bordo.com.au" port="80" query="" referer="-" cookie="-" set-cookie="-" websocket_scheme="-" websocket_protocol="-" websocket_key="-" websocket_version="-" uid="X7MarHBZLmB8Joos1mMIagAAAEs"
2020:11:17-11:34:53 astaro1-1 httpd: id="0299" srcip="64.78.149.164" localip="139.130.139.174" size="254" user="-" host="64.78.149.164" method="GET" statuscode="301" reason="-" extra="-" exceptions="-" time="3752" url="/.well-known/acme-challenge/PftAsVdmkyiDzNJpxfjGkuat-vviRTpiTtZRyuOGPrw" server="internal.bordo.com.au" port="80" query="" referer="-" cookie="-" set-cookie="-" websocket_scheme="-" websocket_protocol="-" websocket_key="-" websocket_version="-" uid="X7MarXBZLmB8Joos1mMIbgAAAGM"
2020:11:17-11:34:54 astaro1-1 httpd[20200]: [authz_blacklist:warn] [pid 20200:tid 3844557680] [client 3.128.26.105:23396] DNSRBL lookup: Match found on fur.cal1.sophosxl.com, referer: internal.bordo.com.au/.../PftAsVdmkyiDzNJpxfjGkuat-vviRTpiTtZRyuOGPrw
2020:11:17-11:34:54 astaro1-1 httpd: id="0299" srcip="3.128.26.105" localip="139.130.139.174" size="279" user="-" host="3.128.26.105" method="GET" statuscode="403" reason="dnsrbl" extra="Client is listed on DNSRBL fur.cal1.sophosxl.com" exceptions="-" time="1725478" url="/.well-known/acme-challenge/PftAsVdmkyiDzNJpxfjGkuat-vviRTpiTtZRyuOGPrw" server="internal.bordo.com.au" port="443" query="" referer="">internal.bordo.com.au/.../PftAsVdmkyiDzNJpxfjGkuat-vviRTpiTtZRyuOGPrw" cookie="-" set-cookie="-" websocket_scheme="-" websocket_protocol="-" websocket_key="-" websocket_version="-" uid="X7MarXBZLmB8Joos1mMIawAAAFQ"
2020:11:17-11:34:55 astaro1-1 httpd[20200]: [authz_blacklist:warn] [pid 20200:tid 3760630640] [client 34.211.6.84:30276] DNSRBL lookup: Match found on fur.cal1.sophosxl.com, referer: internal.bordo.com.au/.../PftAsVdmkyiDzNJpxfjGkuat-vviRTpiTtZRyuOGPrw
2020:11:17-11:34:55 astaro1-1 httpd: id="0299" srcip="34.211.6.84" localip="139.130.139.174" size="279" user="-" host="34.211.6.84" method="GET" statuscode="403" reason="dnsrbl" extra="Client is listed on DNSRBL fur.cal1.sophosxl.com" exceptions="-" time="1435094" url="/.well-known/acme-challenge/PftAsVdmkyiDzNJpxfjGkuat-vviRTpiTtZRyuOGPrw" server="internal.bordo.com.au" port="443" query="" referer="">internal.bordo.com.au/.../PftAsVdmkyiDzNJpxfjGkuat-vviRTpiTtZRyuOGPrw" cookie="-" set-cookie="-" websocket_scheme="-" websocket_protocol="-" websocket_key="-" websocket_version="-" uid="X7MarXBZLmB8Joos1mMIbAAAAF4"
2020:11:17-11:34:55 astaro1-1 httpd[20200]: [authz_blacklist:warn] [pid 20200:tid 3769023344] [client 18.196.96.172:65328] DNSRBL lookup: Match found on fur.cal1.sophosxl.com, referer: internal.bordo.com.au/.../PftAsVdmkyiDzNJpxfjGkuat-vviRTpiTtZRyuOGPrw
2020:11:17-11:34:55 astaro1-1 httpd: id="0299" srcip="18.196.96.172" localip="139.130.139.174" size="279" user="-" host="18.196.96.172" method="GET" statuscode="403" reason="dnsrbl" extra="Client is listed on DNSRBL fur.cal1.sophosxl.com" exceptions="-" time="1416303" url="/.well-known/acme-challenge/PftAsVdmkyiDzNJpxfjGkuat-vviRTpiTtZRyuOGPrw" server="internal.bordo.com.au" port="443" query="" referer="">internal.bordo.com.au/.../PftAsVdmkyiDzNJpxfjGkuat-vviRTpiTtZRyuOGPrw" cookie="-" set-cookie="-" websocket_scheme="-" websocket_protocol="-" websocket_key="-" websocket_version="-" uid="X7MarXBZLmB8Joos1mMIbQAAAF0"

In the WAF settings I've set it to 'No Profile' and unchecked all the boxesin Advanced.

Any suggestions?

Running Release 9.705-3.

Thanks, James.



This thread was automatically locked due to age.
Parents
  • FormerMember
    0 FormerMember

    Hi ,

    Thank you for reaching out to the Community! 

    Do you have any DNAT rule with port 80 or 443 configured on the firewall? 

    Are there any errors in letsencrypt logs? 

    Thanks,

  • 0 in reply to FormerMember

    Thanks for replying H_Patel.

    Have checked, and there are not any DNAT rules for HTTP or HTTPS

    Nothing much in the letsencrypt.log:

    2020-11-17 11:36:17,148:DEBUG:acme.client:Storing nonce: 0103FWRBiUW5rNjSpSOTcspPu9mD3DCRBnUKXSZxBLsZfPo 2020-11-17 11:36:20,152:DEBUG:acme.client:JWS payload: b'' 2020-11-17 11:36:20,154:DEBUG:acme.client:Sending POST request to  {   "protected": "eyJhbGciOiAiUlMyNTYiLCAia2lkIjogImh0dHBzOi8vYWNtZS12MDEuYXBpLmxldHNlbmNyeXB0Lm9yZy9hY21lL3JlZy8xMjMyNTY5OCIsICJub25jZSI6ICIwMTAzRl$   "signature": "iMGhCcIx4Aar0ouV4UWfGTP9XQShop1Lw8y70h1Q8c7xNgGbOnMXCySR0enzLXTYM5JRPUFTycTV7uPXhLQEVy0HU5PAI8cAi-2ZjrbJy22IO8V_8wcATG330gOHIanrzd$   "payload": "" } 2020-11-17 11:36:20,394:DEBUG:urllib3.connectionpool: "POST /acme/authz-v3/8656371242 HTTP/1.1" 200 1373 2020-11-17 11:36:20,395:DEBUG:acme.client:Received response: HTTP 200 Server: nginx Date: Tue, 17 Nov 2020 00:36:20 GMT Content-Type: application/json Content-Length: 1373 Connection: keep-alive Boulder-Requester: 12325698 Cache-Control: public, max-age=0, no-cache Link: <>;rel="index" Replay-Nonce: 0104IoIgPRMG_Oi1SjklAdwmyZDQf-3JA4vOEsyuKkqc5_g X-Frame-Options: DENY Strict-Transport-Security: max-age=604800  {   "identifier": {     "type": "dns",     "value": "internal.bordo.com.au"   },   "status": "invalid",   "expires": "2020-11-24T00:34:50Z",   "challenges": [     {

     "type": "http-01",       "status": "invalid",       "error": {         "type": "urn:ietf:params:acme:error:connection",         "detail": "Fetching  Timeout after co$         "status": 400       },       "url": "",       "token": "PftAsVdmkyiDzNJpxfjGkuat-vviRTpiTtZRyuOGPrw",       "validationRecord": [         {           "url": "",           "hostname": "internal.bordo.com.au",           "port": "80",           "addressesResolved": [             "139.130.139.174"           ],           "addressUsed": "139.130.139.174"         },         {           "url": "",           "hostname": "internal.bordo.com.au",           "port": "443",           "addressesResolved": [             "139.130.139.174"           ],           "addressUsed": "139.130.139.174"         }       ]     }   ] } 2020-11-17 11:36:20,395:DEBUG:acme.client:Storing nonce: 0104IoIgPRMG_Oi1SjklAdwmyZDQf-3JA4vOEsyuKkqc5_g 2020-11-17 11:36:20,396:WARNING:certbot._internal.auth_handler:Challenge failed for domain internal.bordo.com.au 2020-11-17 11:36:20,396:INFO:certbot._internal.auth_handler:http-01 challenge for internal.bordo.com.au 2020-11-17 11:36:20,396:DEBUG:certbot._internal.reporter:Reporting to user: The following errors were reported by the server:  Domain: internal.bordo.com.au Type:   connection Detail: Fetching  Timeout after connect (your server may be slow or

  • 0 in reply to jlbrown

    Looks like the WAF log didn't display properly in my original post. I'll try again:

    2020:11:18-10:03:00 astaro1-1 httpd: id="0299" srcip="64.78.149.164" localip="139.130.139.174" size="257" user="-" host="64.78.149.164" method="GET" statuscode="301" reason="-" extra="-" exceptions="-" time="2134" url="/.well-known/acme-challenge/i3KqQWmy_v_w1R_aZZAe7-klzxw1jEktiym5ErP8g3o" server="internal.bordo.com.au" port="80" query="" referer="-" cookie="-" set-cookie="-" websocket_scheme="-" websocket_protocol="-" websocket_key="-" websocket_version="-" uid="X7RWpHZyd8fEb3cOFvqi6wAAAFY"
    2020:11:18-10:03:00 astaro1-1 httpd: id="0299" srcip="118.209.213.181" localip="139.130.139.174" size="318" user="-" host="118.209.213.181" method="POST" statuscode="200" reason="-" extra="-" exceptions="-" time="168041" url="/4DAction/MAJX_Picking/" server="sapphire.bordo.com.au:444" port="444" query="" referer="">sapphire.bordo.com.au:444/blank.html cookie="-" set-cookie="-" websocket_scheme="-" websocket_protocol="-" websocket_key="-" websocket_version="-" uid="X7RWpHZyd8fEb3cOFvqi6gAAAEs"
    2020:11:18-10:03:01 astaro1-1 httpd[17807]: [authz_blacklist:warn] [pid 17807:tid 3995626352] [client 34.211.6.84:51102] DNSRBL lookup: Match found on fur.cal1.sophosxl.com, referer: internal.bordo.com.au/.../i3KqQWmy_v_w1R_aZZAe7-klzxw1jEktiym5ErP8g3o
    2020:11:18-10:03:01 astaro1-1 httpd: id="0299" srcip="34.211.6.84" localip="139.130.139.174" size="279" user="-" host="34.211.6.84" method="GET" statuscode="403" reason="dnsrbl" extra="Client is listed on DNSRBL fur.cal1.sophosxl.com" exceptions="-" time="2150250" url="/.well-known/acme-challenge/i3KqQWmy_v_w1R_aZZAe7-klzxw1jEktiym5ErP8g3o" server="internal.bordo.com.au" port="443" query="" referer="">internal.bordo.com.au/.../i3KqQWmy_v_w1R_aZZAe7-klzxw1jEktiym5ErP8g3o" cookie="-" set-cookie="-" websocket_scheme="-" websocket_protocol="-" websocket_key="-" websocket_version="-" uid="X7RWo3Zyd8fEb3cOFvqi5gAAAEI"
    2020:11:18-10:03:01 astaro1-1 httpd[17807]: [authz_blacklist:warn] [pid 17807:tid 3777416048] [client 3.128.26.105:52576] DNSRBL lookup: Match found on fur.cal1.sophosxl.com, referer: internal.bordo.com.au/.../i3KqQWmy_v_w1R_aZZAe7-klzxw1jEktiym5ErP8g3o
    2020:11:18-10:03:01 astaro1-1 httpd: id="0299" srcip="3.128.26.105" localip="139.130.139.174" size="279" user="-" host="3.128.26.105" method="GET" statuscode="403" reason="dnsrbl" extra="Client is listed on DNSRBL fur.cal1.sophosxl.com" exceptions="-" time="2138688" url="/.well-known/acme-challenge/i3KqQWmy_v_w1R_aZZAe7-klzxw1jEktiym5ErP8g3o" server="internal.bordo.com.au" port="443" query="" referer="">internal.bordo.com.au/.../i3KqQWmy_v_w1R_aZZAe7-klzxw1jEktiym5ErP8g3o" cookie="-" set-cookie="-" websocket_scheme="-" websocket_protocol="-" websocket_key="-" websocket_version="-" uid="X7RWo3Zyd8fEb3cOFvqi5wAAAFw"
    2020:11:18-10:03:02 astaro1-1 httpd[17807]: [authz_blacklist:warn] [pid 17807:tid 3962055536] [client 18.196.96.172:51254] DNSRBL lookup: Match found on fur.cal1.sophosxl.com, referer: internal.bordo.com.au/.../i3KqQWmy_v_w1R_aZZAe7-klzxw1jEktiym5ErP8g3o
    2020:11:18-10:03:02 astaro1-1 httpd: id="0299" srcip="18.196.96.172" localip="139.130.139.174" size="279" user="-" host="18.196.96.172" method="GET" statuscode="403" reason="dnsrbl" extra="Client is listed on DNSRBL fur.cal1.sophosxl.com" exceptions="-" time="2131670" url="/.well-known/acme-challenge/i3KqQWmy_v_w1R_aZZAe7-klzxw1jEktiym5ErP8g3o" server="internal.bordo.com.au" port="443" query="" referer="">internal.bordo.com.au/.../i3KqQWmy_v_w1R_aZZAe7-klzxw1jEktiym5ErP8g3o" cookie="-" set-cookie="-" websocket_scheme="-" websocket_protocol="-" websocket_key="-" websocket_version="-" uid="X7RWpHZyd8fEb3cOFvqi6AAAAEY"
    2020:11:18-10:03:06 astaro1-1 httpd: id="0299" srcip="64.78.149.164" localip="139.130.139.174" size="113" user="-" host="64.78.149.164" method="GET" statuscode="200" reason="-" extra="-" exceptions="-" time="2568453" url="/.well-known/acme-challenge/i3KqQWmy_v_w1R_aZZAe7-klzxw1jEktiym5ErP8g3o" server="internal.bordo.com.au" port="443" query="" referer="">internal.bordo.com.au/.../i3KqQWmy_v_w1R_aZZAe7-klzxw1jEktiym5ErP8g3o" cookie="-" set-cookie="-" websocket_scheme="-" websocket_protocol="-" websocket_key="-" websocket_version="-" uid="X7RWqHZyd8fEb3cOFvqi7AAAAFA"
  • 0 in reply to jlbrown

    Did you solve the problem? If yes, how did you do it?

  • FormerMember
    +1 FormerMember in reply to jlbrown

    Hi ,

    Could you please check if you have country blocking configured under Network Protection > Firewall > Country Blocking? 

    If it is configured, please check out the following community post and configure the country blocking exception. 

    Thanks,

  • +1 in reply to FormerMember

    Thanks that worked.

    For anyone else who has the problem, these are the four exceptions I set up:

    Thanks again everyone!

    James.

  • 0 in reply to FormerMember

    I don't think that's his problem because he want's to through the WAF.

    @jlbrown ... have a look into the LetsEnrypt Log ... I have tried similar. The problem was and is still, that the WAF "changes" the challenge certbot wanna see. I see in my log, that an HTML DOCTYPE is added in the second phase of validation. So the validation fails.

    Please, can you post your LE log-file?

  • 0 in reply to tomcek

    Not sure which bit of the log you want to see. This bit seems to indicate the challenge response is accepted?

    Bit scared of posting the whole log file in case I reveal something I shouldn't!

    James.

Reply Children
No Data
Unfiltered HTML