Thread Info
  • State Verified Answer
  • Locked Locked
  • Replies 4 replies
  • Subscribers 6 subscribers
  • Views 13085 views
  • Users 0 members are here
Options
Suggested
This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

Bug - Improper headers for blocked content

Unknown Unknown1

Dear all,

While toying around with cURL and Sophos Anti-Virus, I noticed that the Web Protection service:

▸ Returns a 403 Forbidden error when the domain's reputation is deemed unsuitable

▸ Returns a 200 OK error when the content is blocked

Compare, for example:

▸ Loading the Sophos malware-blocking test page

$ /usr/bin/curl --compressed "http://sophostest.com/malware/" -sI
    HTTP/1.1 403 Forbidden
    Content-Length: 6865
    Content-Type: text/html; charset=UTF-8
    Cache-Control: no-cache
    Connection: close
    Proxy-Connection: close

▸ Loading the Eicar.org pseudo-virus

 
/usr/bin/curl --compressed “www.eicar.org/.../eicar.com.txt" -sI
    HTTP/1.1 200 OK
    Date: Fri, 03 Jul 2015 12:20:55 GMT
    Server: Apache
    Content-disposition: attachment; filename="eicar.com.txt"
    Cache-control: private
    Content-length: 68
    Content-Type: application/octet-stream
 
This seems to be a bug, as both attempts result in a Sophos-generated error page being displayed.
:1021180


This thread was automatically locked due to age.
  • 0

    Hey Francois

    I did some investigation into this, and it's actually expected behaviour. 

    What you're actually demonstrating is the functionality of two different features of web protection:

    The first setting here blocks access to websites based on their reputation. The second blocks websites based on the content of the site, so it will block access to a page if it contains malicious content. 

    So when you do a curl against http://www.eicar.org/download/eicar.com.txt, it's really only looking at http://www.eicar.org/download, which we don't block. If you have the second web protection feature enabled, you will still get a block page if you attempt to browse to the file eicar.com.txt file, because the content is malicious.

    Just to test this, I disabled the second feature above and attempted to download the eicar file, and it worked... although since I kept my on-access scanning enabled, it was blocked as soon as it was written to hard the hard drive. 

    Anyway, I hope that answers your question, but feel free to ask me anything if you're still unsure.

    :1021189
  • 0

    Thank you for your kind reply, Serra.

    These are indeed two distinct features, and I do understand that they behave in a different way. It does seem strange, however, to display similar results to the user, namely the block page, while returning two different sets of headers to the browsers or scripts that bump into the error. This makes it difficult to evaluate the situation programatically, and I can imagine quite a few scenarios where it could be confusing.

    It's a minor issue, of course…

    :1021195
  • 0

    I'm curious to hear more about your thoughts on this. Can you give me some examples of when you think this implementation of the block page could be confusing? I can't make any promises but I can definitely pass your ideas onto to the development team.

    :1021214
  • 0

    Hello Serra,

    As the go-to admin for an ever-growing circle of friends and family, I often whip up small scripts to help in troubleshooting and debugging network issues. These scripts perform a few dig and curl operations, compare the results against what would be expected in a known-good setup, and attempt to perform what changes can be done automatically.

    One of my steps is to test whether Sophos Web Protection is active. I actively recommend SAV 9.x to friends, but I find they sometimes change the configuration. (People with slow DNS will find Sophos Web Protection to be all but unusable and to have a paralysing effect on the machine while it is totally transparent to others. For example, OpenDNS Umbrella + Sophos = Total lockdown.)

    Testing whether reputation filtering is in place is as simple as:

    /usr/bin/curl "http://sophostest.com/malware/" -sI | grep -Eq "^HTTP/1\.1\ 403\ Forbidden\r$"

    However, testing for content filtering is very messy:

    /usr/bin/curl "www.eicar.org/.../eicar.com.txt" -s | grep -Eq "Sophos\ Block\ Page"

    The command is just as short, but we must download the entire page instead of just the headers, which takes longer and uses more resources. We must also use grep to parse the resulting [physique, build, form, frame, figure, anatomy, skeleton, trunk, torso]*, which is prone to failure, especially should Sophos ever decide to change the page format. The headers preceding the content appear to be those of the server being blocked. For example, in the case of the EICAR page above, I see the following, which looks suspiciously like what EICAR themselves send back. (I doubt that Sophos runs a full Apache install locally, or does it?)

    HTTP/1.1 200 OK
Date: Tue, 07 Jul 2015 05:53:17 GMT
Server: Apache
Content-disposition: attachment; filename="eicar.com.txt"
Cache-control: private
Content-length: 68
Content-Type: application/octet-stream

    Both commands result in error pages. From a purely logical standpoint, it seems the headers should always match the content. When the proxy serves a block page, it "forbids" loading the requested content just as much as when reputation filtering activates. The fact that the underlying mechanisms are different changes little to the actual result. Furthermore, since no browser on OS X offers "Custom Error Pages" by default (à la Internet Explorer of yore), to my knowledge at least, I feel that it would be quite safe to use the proper response codes.

    I realise that curl and scripts are edge cases, but I am certainly not the only Sophos user who noodles around in Terminal…

    * — Well, the you-guess-what anyway. The forum's profanity filter is unhappy about that word.

    :1021243
Unfiltered HTML