Thread Info
  • State Verified Answer
  • Locked Locked
  • Replies 9 replies
  • Subscribers 5 subscribers
  • Views 17735 views
  • Users 0 members are here
Options
Suggested
This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

Web filter log only shows Input/Output Errors

Andrew Parker

I'm running UTM9.502-4. I've been trying to fine tune some web filtering exceptions after enabling Scan and Decrypt in Transparent mode. This caused a lot of iOS apps not work, which I've gathered is a normal occurrence because of cert pinning in the apps.

I've been looking at the log to figure out what to add an exception for, but all I'm seeing are Input/Output errors like this:

2017:08:28-16:28:29 firewall httpproxy[5721]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="read_request_headers" file="request.c" line="1586" message="Read error on the http handler 2407 (Input/output error)"

2017:08:28-16:28:30 firewall httpproxy[5721]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="read_request_headers" file="request.c" line="1586" message="Read error on the http handler 2407 (Input/output error)"

2017:08:28-16:28:30 firewall httpproxy[5721]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="read_request_headers" file="request.c" line="1586" message="Read error on the http handler 2407 (Input/output error)"

2017:08:28-16:28:35 firewall httpproxy[5721]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="read_request_headers" file="request.c" line="1586" message="Read error on the http handler 2407 (Input/output error)"

2017:08:28-16:28:35 firewall httpproxy[5721]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="read_request_headers" file="request.c" line="1586" message="Read error on the http handler 2407 (Input/output error)"

2017:08:28-16:28:36 firewall httpproxy[5721]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="read_request_headers" file="request.c" line="1586" message="Read error on the http handler 2407 (Input/output error)"

2017:08:28-16:28:36 firewall httpproxy[5721]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="read_request_headers" file="request.c" line="1586" message="Read error on the http handler 2458 (Input/output error)"

2017:08:28-16:28:36 firewall httpproxy[5721]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="read_request_headers" file="request.c" line="1586" message="Read error on the http handler 2458 (Input/output error)"

2017:08:28-16:28:36 firewall httpproxy[5721]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="read_request_headers" file="request.c" line="1586" message="Read error on the http handler 2458 (Input/output error)"

2017:08:28-16:28:36 firewall httpproxy[5721]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="read_request_headers" file="request.c" line="1586" message="Read error on the http handler 2458 (Input/output error)"

2017:08:28-16:28:36 firewall httpproxy[5721]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="read_request_headers" file="request.c" line="1586" message="Read error on the http handler 2407 (Input/output error)"

2017:08:28-16:28:36 firewall httpproxy[5721]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="read_request_headers" file="request.c" line="1586" message="Read error on the http handler 2407 (Input/output error)"

2017:08:28-16:28:36 firewall httpproxy[5721]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="read_request_headers" file="request.c" line="1586" message="Read error on the http handler 2407 (Input/output error)"

2017:08:28-16:28:41 firewall httpproxy[5721]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="read_request_headers" file="request.c" line="1586" message="Read error on the http handler 2452 (Input/output error)"

2017:08:28-16:28:42 firewall httpproxy[5721]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="read_request_headers" file="request.c" line="1586" message="Read error on the http handler 2452 (Input/output error)"

2017:08:28-16:28:42 firewall httpproxy[5721]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="read_request_headers" file="request.c" line="1586" message="Read error on the http handler 2452 (Input/output error)"

2017:08:28-16:28:42 firewall httpproxy[5721]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="read_request_headers" file="request.c" line="1586" message="Read error on the http handler 2452 (Input/output error)"

2017:08:28-16:28:42 firewall httpproxy[5721]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="read_request_headers" file="request.c" line="1586" message="Read error on the http handler 2407 (Input/output error)"

2017:08:28-16:28:42 firewall httpproxy[5721]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="read_request_headers" file="request.c" line="1586" message="Read error on the http handler 2407 (Input/output error)"

2017:08:28-16:28:42 firewall httpproxy[5721]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="read_request_headers" file="request.c" line="1586" message="Read error on the http handler 2407 (Input/output error)"

2017:08:28-16:28:42 firewall httpproxy[5721]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="read_request_headers" file="request.c" line="1586" message="Read error on the http handler 2458 (Input/output error)"

2017:08:28-16:28:47 firewall httpproxy[5721]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="read_request_headers" file="request.c" line="1586" message="Read error on the http handler 2407 (Input/output error)"

2017:08:28-16:28:47 firewall httpproxy[5721]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="read_request_headers" file="request.c" line="1586" message="Read error on the http handler 2407 (Input/output error)"

I had to hook my iPhone up to a Burp proxy to discover that an exception for inbox.google.com was required for the gmail app to work.

This is making it near impossible to figure out the exceptions I need to add. Is this a bug?



This thread was automatically locked due to age.
  • 0

    I have simular problems with the 9.413 and connections from the Sage 200c Software. Also Our second Security Software has problems to reach his Cloud Services. If I say the software/the system, use Sophos UTM as Proxy, all works.

    So I think, this is a bug.

    My problem is: I am not able to add for all an exclude rules because in the log file I can not see which URL was called.

  • 0

    Hi Andrew,
    we had similar entries in the Web Protection Live Log.
    In addition, our CPU utilization on both cores increased to ~99.8% (SG125 with FW v9.502-4).
    The evaluation with "top" over the CLI resulted in 150 - 180% CPU usage for httpproxy.

    Our solution was to import the UTM certificate into Firefox. (We use Web Proxy with authentication and SSL scan).
    After importing the certificate the UTM behaved normal again and the output of top over the CLI also showed moderate values (around 30 - 60%) for httpproxy.

    Furthermore, we have defined an exception for the corresponding service (on localhost) in the Internet options.


    Many greetings

    DSC

  • 0 in reply to KaiDietrich

    KaiDietrich said:

    I have simular problems with the 9.413 and connections from the Sage 200c Software. Also Our second Security Software has problems to reach his Cloud Services. If I say the software/the system, use Sophos UTM as Proxy, all works.

    So I think, this is a bug.

    My problem is: I am not able to add for all an exclude rules because in the log file I can not see which URL was called.

     

     

    Hi KaiDietrich,

    try the following:
    1. look over the CLI with "top" for the load-causing process. If it's the httpproxy as in our case:
    2. run a daily report via WebAdmin
    3. look under "Network Usage -> Top10 Clients" for the corresponding host, which causes the most traffic
    4. further troubleshooting on the corresponding host

    Do you use SSL scan over web proxy?
    If so, import the UTM certificate into the certificate manager on the host (or alternatively into the alternate browser).

     

    *** EDIT ***

    Have you entered the UTM as the default gateway on the system with the Sage software?
    If yes, you have checked the FW-Livelog and the WebFilter Livelog, if e. g. Say you want to set up an internet connection?

    What does it say?

    Have you created a test version of an "any-FW-rule" for the corresponding services only for the system on the internet (Host -> any -> Internet IPv4)?

    *** EDIT ends ***

    Many greetings

    DSC

  • 0 in reply to Daniel Schwechheimer

    Thanks for the reply Daniel.  I have already imported the certificate on my iPhone.  I enabled the trust settings, but some apps still don't work because of certificate pinning.  So I need to add exceptions for those urls, but the log is not showing any useful information when I'm using my iPhone.  My CPU usage is low so that's not a problem.

  • 0 in reply to Andrew Parker

    Andrew Parker said:

    Thanks for the reply Daniel.  I have already imported the certificate on my iPhone.  I enabled the trust settings, but some apps still don't work because of certificate pinning.  So I need to add exceptions for those urls, but the log is not showing any useful information when I'm using my iPhone.  My CPU usage is low so that's not a problem.

    Hi Andrew,
    sorry I can't help you anymore.

    Maybe Bob reads this thread and has some useful suggestions.

    Many greetings

    DSC

  • 0

    I have seen this symptom on sites that use sha1 certificates, which are insecure.  If you want to connect anyway, you create an exceptiom to skip certificate checks.

    Use openssl.com server test page to see whether the site is configured securely or not.   Most ssl vendors have ssl testers which are wuicker but less comprehensive.   I use a mixture of both.  UTM logs are minimally useful for explaining why a certificate is problematic.

  • 0 in reply to DouglasFoster

    The problem is that I don't know what site to add an exception for because the log does not show the url that is being blocked.  It's just showing the Input/Output error.

  • 0 in reply to Daniel Schwechheimer

    Thank you for your Reply.

    But I don't think, this is the problem. The SOPHOS WEB Protection Cetificate is install via GPO on all Systems. So if Sage does not use his own Certificate Store, the Connection is be Trust.

    We also have no High LOAD on the UTM and yes, the UTM is the default gateway of the Network.

    The Idea to open the Firewall completly for one specific host is a good Idea, if the Software use other Ports as 80 and 443. But also here, the Entrys in the Log File shows me, that there is a problem with analyze the HTTP Headers. And a Firewall Rule don't Help if this is realy the Problem, because the Transparent Proxy Rule (Get all Communication to Port 80/443) is before the User Firewall Rules.

  • +1 in reply to Andrew Parker

    I found something that helps.  In my Filter Action I had only checked Log Blocked pages.  When I also enabled Log Accessed pages I started to see some more useful things in the Web Filter Log.

Unfiltered HTML