SSL scanning implementation / Best practise (errorhandling....)

Question

I&acute;m interesting in your experiences with activating the ssl scanning feature. I implemented and activated it in an environment serving about 4000 users.Just to imagine, there are (per day) 11820 unique domains and 2300 unique users. 
 Many different webpages are visited and my experience is, that many sites are broken or not visible after activation. I have to create a lots of exceptions for ssl scanning and some for the certificate check. Sometimes I face problems with certificates, that seems to be ok (no problems with direct internet connection, works good with the same browser). What can cause this? Could it be, that the utm doesn&acute;t know all public trusted root CAs? 
 So I&acute;m interested in your experience, how to handle all this... Do you completely disable the certificate checks (because you might think the enduser would skip all the security warnings anyway? ) ? Or do you also prefer to create exceptions? 
 
 Ciao 
 Sebastian

RichBaldry · Answer

You can install the intermediate certificates, if you can get hold of them. 
 Browsers mask the problem of poorly-configured web servers by caching intermediate certificates that they download from well-behaved servers. So in the case cited by the OP, portal.bibserve.com has a cert signed by "Symantec Class 3 Secure Server CA - G4", which in turn is signed by a Verisign root cert. If you try to access this site on a brand new machine with a brand new browser, it will probably fail. However, as soon as you go to another site with a certificate signed by the same Symantec intermediate cert, your browser will store the intermediate cert in a cache. The next time you try to visit the broken site, the browser says "Hey, I know about that intermediate cert" and uses the cached copy to validate it. 
 I found Firefox was the easiest way to capture the intermediate cert, save it, and install it to the UTM. I run a Mac, and both Safari and Chrome use OS X's built-in keychain manager, which doesn't seem to expose cached certs. 
 In Firefox, I could click on the lock icon, go to More Information, go to the Security tab, go to View Certificate, Click on the Details tab, select the line under 'Certificate Hierarchy' for the Symantec Class 3 Secure Server CA, click Export... and save the certificate. This saved file, given a .pem extension, can be loaded into the UTM under Web Protection > Filtering Options > HTTPS CAs > Local verification CAs. 
 To save the bother in this case, here is the contents of the cert used by portal.bibserve.com. Copy and paste it into a text file, save it with a .pem extension and it should import into your UTM: 
 -----BEGIN CERTIFICATE----- MIIFODCCBCCgAwIBAgIQUT+5dDhwtzRAQY0wkwaZ/zANBgkqhkiG9w0BAQsFADCB yjELMAkGA1UEBhMCVVMxFzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMR8wHQYDVQQL ExZWZXJpU2lnbiBUcnVzdCBOZXR3b3JrMTowOAYDVQQLEzEoYykgMjAwNiBWZXJp U2lnbiwgSW5jLiAtIEZvciBhdXRob3JpemVkIHVzZSBvbmx5MUUwQwYDVQQDEzxW ZXJpU2lnbiBDbGFzcyAzIFB1YmxpYyBQcmltYXJ5IENlcnRpZmljYXRpb24gQXV0 aG9yaXR5IC0gRzUwHhcNMTMxMDMxMDAwMDAwWhcNMjMxMDMwMjM1OTU5WjB+MQsw CQYDVQQGEwJVUzEdMBsGA1UEChMUU3ltYW50ZWMgQ29ycG9yYXRpb24xHzAdBgNV BAsTFlN5bWFudGVjIFRydXN0IE5ldHdvcmsxLzAtBgNVBAMTJlN5bWFudGVjIENs YXNzIDMgU2VjdXJlIFNlcnZlciBDQSAtIEc0MIIBIjANBgkqhkiG9w0BAQEFAAOC AQ8AMIIBCgKCAQEAstgFyhx0LbUXVjnFSlIJluhL2AzxaJ+aQihiw6UwU35VEYJb A3oNL+F5BMm0lncZgQGUWfm893qZJ4Itt4PdWid/sgN6nFMl6UgfRk/InSn4vnlW 9vf92Tpo2otLgjNBEsPIPMzWlnqEIRoiBAMnF4scaGGTDw5RgDMdtLXO637QYqzu s3sBdO9pNevK1T2p7peYyo2qRA4lmUoVlqTObQJUHypqJuIGOmNIrLRM0XWTUP8T L9ba4cYY9Z/JJV3zADreJk20KQnNDz0jbxZKgRb78oMQw7jW2FUyPfG9D72MUpVK Fpd6UiFjdS8W+cRmvvW1Cdj/JwDNRHxvSz+w9wIDAQABo4IBYzCCAV8wEgYDVR0T AQH/BAgwBgEB/wIBADAwBgNVHR8EKTAnMCWgI6Ahhh9odHRwOi8vczEuc3ltY2Iu Y29tL3BjYTMtZzUuY3JsMA4GA1UdDwEB/wQEAwIBBjAvBggrBgEFBQcBAQQjMCEw HwYIKwYBBQUHMAGGE2h0dHA6Ly9zMi5zeW1jYi5jb20wawYDVR0gBGQwYjBgBgpg hkgBhvhFAQc2MFIwJgYIKwYBBQUHAgEWGmh0dHA6Ly93d3cuc3ltYXV0aC5jb20v Y3BzMCgGCCsGAQUFBwICMBwaGmh0dHA6Ly93d3cuc3ltYXV0aC5jb20vcnBhMCkG A1UdEQQiMCCkHjAcMRowGAYDVQQDExFTeW1hbnRlY1BLSS0xLTUzNDAdBgNVHQ4E FgQUX2DPYZBV34RDFIpgKrL1evRDGO8wHwYDVR0jBBgwFoAUf9Nlp8Ld7LvwMAnz Qzn6Aq8zMTMwDQYJKoZIhvcNAQELBQADggEBAF6UVkndji1l9cE2UbYD49qecxny H1mrWH5sJgUs+oHXXCMXIiw3k/eG7IXmsKP9H+IyqEVv4dn7ua/ScKAyQmW/hP4W Ko8/xabWo5N9Q+l0IZE1KPRj6S7t9/Vcf0uatSDpCr3gRRAMFJSaXaXjS5HoJJtG QGX0InLNmfiIEfXzf+YzguaoxX7+0AjiJVgIcWjmzaLmFN5OUiQt/eV5E1PnXi8t TRttQBVSK/eHiXgSgW7ZTaoteNTCLD0IX4eRnh8OsN4wUmSGiaqdZpwOdgyA8nTY Kvi4Os7X1g8RvmurFPW9QaAiY4nxug9vKWNmLT+sjHLF+8fk1A/yO0+MKcc= -----END CERTIFICATE----- 
 This would be a neat feature to add to the UTM, I agree. I've added it to the Uservoice forum here: http://feature.astaro.com/forums/17359-utm-formerly-asg-feature-requests/suggestions/12887928-cache-intermediate-https-issuer-certificates 
 Cheers, Rich

DouglasFoster · Answer

I use two SSL site checkers. the one from my certificate vendor, Thawte, and the one from SSLLABS.com that was mentioned previously. Thawte is part of Symantec. The URL is https://cryptoreport.thawte.com/checker/views/certCheck.jsp 
 When the Thawte utility sees a missing intermediate for a Symantec or Thawte certificate, it will provide a link to download it. You install it within UTM using this click sequence: Web Protection... Filtering Options... HTTPS CAs...Then choose the "Upload Local CAs" from the "Verification CAs" section of the page. This works, even though the certificate is not a CA certificate. 
 I post the Thawte link reluctantly. As a courtesy to them, you should use the SSL checker provided by your certificate vendor first. 
 I have determined that Godaddy only issues binary intermediate certificates. UTM cannot load these. 
 I don't have any information for other certificate sources. 
 The SSL Labs test utility is more comprehensive and slower, so I only use it when the Thawte site is insufficient. Ssllabs has the advantage of identifying all of the IP addresses associated with a DNS name, including both IPV4 and IPV6. If you have inconsistent results, it may be because the remote site uses more than one IP address, and the servers are configured differently. 
 SSL Labs provides better insight into certificate chain problems, particularly "contains anchor" This means that the site is sending a root certificate as part of the chain, which is incorrect. UTM will reject the certificate chain if anything is self-signed, which is always the case for a root certificate, which is why the root has to be preinstalled on your PC to be trusted. 
 As I have explained elsewhere, I parse yesterday's log files to find all of the certificate errors, then use the SSL checker sites to identify the actual problem, then use WHOIS to idenify someone at the organization to contact, then use Microsoft Word mail-merge to send an email to all of the sites for which I have a contact. If I cannot find a contact with WHOIS and the entity is obviously legitimate, I try to make contact using their website "Contact Us" page. I usually have about 10 URLs per day, on a large population but not as large as yours. Most sites are very responsive. 
 Parsing the log files to identify missing certificates is complicated. I have posted the details as several replies to "UTM best practise guide for strict webfiltering", which is at this URL https://community.sophos.com/products/unified-threat-management/f/55/p/74489/286777#286777 
 This notification process has saved me from having to create an exception list that is so long that I have to worry about performance problems. Besides, why should I weaken my defenses just because someone else is messed up?