Thread Info
  • State Not Answered
  • Locked Locked
  • Replies 12 replies
  • Subscribers 2 subscribers
  • Views 39814 views
  • Users 0 members are here
Options
Suggested
This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

mails rejected "server certificate not trusted" (dnssec behind utm)

Mike Bruderer

Hope someone with some expertise can jump in:

I have a personal mail server behind an UTM 9 Home (Firmware 9.413-4) and just enabled E-Mail Protection on the UTM (Bridge Mode - works fine). While Email filtering etc. is working extremely good (Spam is blocked, SMTP Logs looks fine etc.), some certificate chain seems to be broken. some third party mail servers do not deliver incoming AND outgoing mails with the error on BOTH sides:

Server certificate not trusted / Diagnostic-Code: X-Postfix; Server certificate not trusted

The mail server handles all the certificates and even DNSSEC is implemented. The setup of the mail server itself is fine and it worked before deployed of the UTM. Having said that, the mail server tests do actually show (after deployment) an error (e.g. on https://de.ssl-tools.net/mailservers/; "unknown authority" as self signed cert is on UTM). It seems to check now the UTM internal TLS cert, and not anymore the original one directly on the mails server. While this makes in some way sense (as UTM is acting as a "MiM") i need to get rid of this problem, as a whole bunch of mails do not get delivered or received. My settings in E-Mail Protection -> SMTP are:

- Simple Mode

- Routing Tab: Domain: mydomain.xy / Route by: Static Host / Host List: (internal) IP of mail server / Recipient Verification: With callout

- Relaying Tab: Upstream Hosts/Networks: (internal) IP of mail server / Allowed Hosts/Networks: (internal) / Scan relayed (outgoing) messages: checked

- Advanced Tab: Use transparent mode: checked / TLS Settings: Local Cert (selected) / Advanced Settings: SMTP Hostname: mydomain.xy BATV: unset etc. / Smarthost Settings: (no smarthost)

Is it maybe possible to use E-Mail Protection without using a certificate on UTM? Means: Can i deploy UTM Mail Protection without relying on UTM certs?

Anyone can jump in here? Thx!



This thread was automatically locked due to age.
  • 0 in reply to Mike Bruderer

    I would think that if both ends are DNSSEC-enabled, that there would be some mutual validation on both incoming and outgoing connections.   Consequently, I am surprised that you only have problems one direction.

    Sometimes when debugging, it is important to go back to basics.   At the risk of sounding trite, here are some things to check:

    Send to a non-DNSSEC recipient and check the email header logs.

    1) Is outbound mail going out on the IP address that you expect?   

    2) Are the assserted SMTP HELO hostname, certificate name, and reverse-DNS identities all consistent and what you intend?  

    3) Are you using a wildcard or a named certificate?   If a wildcard, are you incorrectly trying to use it to cover multiple levels?  

    (NOTE:  "*.mycompany.com" is not valid for "host1.division1.mycompany.com")

    Use a downloaded OpenSSL kit to test SMTP with STARTTLS from the outside:

    3) Do you get the certificate chain that you expect?  

    4) Does it connect?  What ciphersuite is used for the connection, and is it a current one?

    Hope this helps.

    Miscellaneous, which requires contacting a communication partner:

    4) For the sites that are rejecting you, do they recognize and trust your Certificate Authority, or is the root certificat the problem?

  • 0 in reply to DouglasFoster

    Really appreciate your efforts . As Conan Doyle once said: "There is nothing more deceptive than an obvious fact." Your approach is fully justified. 

    Although with the risk to complicate things, i need to give again the following information: the mail server (mail.mailserver.com) is behind the UTM (UTM in bridge mode and SMTP in transparent mode with host name mailserver.com).

    To answer your question:

    1) Is outbound mail going out on the IP address that you expect?  
    Yes. The IP Address is the mail servers IP.

    2) Are the assserted SMTP HELO hostname, certificate name, and reverse-DNS identities all consistent and what you intend?  
    No. I think here lies a problem. I will get into that more detailed below. (*)

    3) Are you using a wildcard or a named certificate?   If a wildcard, are you incorrectly trying to use it to cover multiple levels?  
    No wildcard. Let`s Encrypt as a CA for mail.mailserver.com, mailserver.com and www.mailserver.com (3 single certs).

    Use a downloaded OpenSSL kit to test SMTP with STARTTLS from the outside:
    3) Do you get the certificate chain that you expect?  
    SSL Tool gives all (three) green light and no errors or warnings.

    4) Does it connect?  What ciphersuite is used for the connection, and is it a current one?
    Yes. Ciphersuite: TLSv1.2 ECDHE-RSA-AES256-GCM-SHA384


    (*) Going back to point two again:

    We remember that I replicated (completely) the certs for mail.mailserver.com from the mail server to UTM and UTM is offering those now. This helped me to get rid of the error "server certificate not trusted" and the error changed to "server certificate not verified". Indeed, outbound mails to Email providers with implemented DANE SMTP validation started to get through, but no inbound messages from them ever arrived ("server certificate not verified"). Btw it works perfectly fine in both directions with Email providers that do not have DANE SMTP validation).

    Having said that, I continued with some IP checks for sending mail servers and sent an email from the mail server over to MultiRBLValli. FCrDNS Test all fine (green), Sender Address Tests all fine (green), but with "HELO/EHLO" Tests I run into a FAIL:

    The HELO/EHLO string "mailserver.com" is a valid host- or domainname OK
    IP Addresses for "mailserver.com"  :   99.999.99.999 OK
    At least one IP address of the DNS lookup for "mailserver.com" matches to the connecting IP 99.999.99.999 OK
    The HELO/EHLO string "mailserver.com" doesn't match to one of the names in the rDNS from connecting IP Failed
     
    While not sure if this is the final problem, it could be a lead. So whats happening here?

    The rDNS 99.999.99.999 points to the mail server behind the UTM at mail.mailserver.com - that's correct imho. When using MxToolbox, it safely says for mailserver.com that SMTP Reverse DNS is fine - 99.999.99.999 resolves to mail.mailserver.com.

    But when using the MultiRBLValli to check outbound mails directly, the "fail" above shows up. Found out UTM hostname is set to mail.mailserver.com (so the certs do match) and in E-Mail Protection (Advanced - Advanced Settings - SMTP hostname), the hostname is set to mailserver.com (and not: mail.mailserver.com).

    Interim Conclusion: There is a mismatch between the rDNS entry and the "received from" in the mail header.Guess this is because UTM is "intercepting", acts as a MTA as we know and modifies the information.

    No solution so far: Can not change the SMTP hostname on UTM to mail.mailserver.com - this results immediately in an "Undelivered Mail Returned to Sender" answer from the mail server MTA: "mail for emailaddress@email.com loops back to myself". It could be that third party mail servers who enabled DNSSEC do strict checking and this mismatch could result in the "server certificate not verified". But this goes beyond my knowledge for sure. So still stuck with the rejected mails.

Unfiltered HTML