Thread Info
  • State Not Answered
  • Locked Locked
  • Replies 10 replies
  • Subscribers 3 subscribers
  • Views 10921 views
  • Users 0 members are here
Options
Suggested
This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

Web Filtering whitelist weird issue

MikkoCaldara

Hello,

We are using Sophos UTM 9.403-4 on AWS, AMI ID: ami-d1eb8ba6

Sophos acts as NAT for our EC2 instances outbound traffic, within an Amazon VPC.
UTM is configured with Web Filtering in transparent mode.
We have a list of domains that we want to whitelist to be accessible from our EC2 instances.
The problem we are seeing is described below.
Some of the Web requests are coming through to IPs rather than domains. This breaks our whitelisting, since those requests are blocked.
The domains for which this is happening are: 
- newrelic.com
- rubygems.org and a few more
I'm pretty sure the problem is not the client's software, but rather something weird happening in UTM. (It would not make sense to write software that resolves DNS names to IPs and then invokes the IPs directly. Also, SSL validation would fail miserably!)
An example log entry going to an IP directly:
2016:07:15-13:52:46 uat1-nat-a httpproxy[4930]: id="0001" severity="info" sys="SecureWeb" sub="http" name="http access" action="pass" method="CONNECT" srcip="172.17.94.57" dstip="50.31.164.164" user="" group="" ad_domain="" statuscode="200" cached="0" profile="REF_HttProContaInterNetwo (Web-Profile)" filteraction="REF_HttCffAllowwebti2 (Allow-WebTier)" size="5205" request="0x1517b800" url="https://50.31.164.164/" referer="" error="" authtime="0" dnstime="2" cattime="329" avscantime="0" fullreqtime="399648" device="0" auth="0" ua="" exceptions=""

I had to manually whitelist 50.31.164.164, which belongs to newrelic.com. This is not really viable solution as these IP change often...

Have you seen this issue before? Would you be able to point me in the right direction?
Thanks a lot for your time.
Cheers
Mikko


This thread was automatically locked due to age.
Parents
  • 0

    Web filtering, including whitelisting, works on the URL that is in the GET.

    The transparent mode skiplist works at a firewall level on the actual hosts/ips.  If you create a URL Group object for the domain and skip it, does that resolve your problem?

  • 0 in reply to Michael Dunn

    I can't seem to find how to create URL Group objects. Using Sophos UTM 9.404

    Do you mean DNS Group to be used in the firewall? We attempted that but it won't work because it does not support wildcards. i.e. we need to whitelist *.amazonaws.com

    Thanks

  • 0 in reply to MikkoCaldara

    Sorry I meant DNS Group.

    amazonaws is used by a large number of companies doing a large number of things.  Whitelisting them would effectively put hundreds of website to be whitelisted.  So I have two questions.

    1) is whitelisting really what you want to do, what problems are you trying to solve

    2) most things that are hosted on amazon have other dns entries other than the ones at amazonaws.com.  For example sophos has sandbox.sophos.com which is hosted in EC2.  If you resolve the IPs, and then resolve those IPs back to hostnames you will find they are in amazonaws.com.  But if I wanted to do something with the IPs associated with that service, I would use the IPs that come from the dns resolution of sandbox.sophos.com, I wouldn't want to do something to everything host by anyone in EC2.

  • 0 in reply to Michael Dunn

    1) yes it is, we need to filter all outgoing traffic for security reasons.

    2) amazonaws.com was just an example, we have more domains that we notice are producing this weird behaviour, for example newrelic.com and rubygems.org

    Reproducing is very easy, just installing aws cli tools and invoking aws s3 ls will result in a web request to some random Amazon IP, rather than a hostname. This is very annoying. For now we wrote some regex to capture a lot of Amazon CIDR ranges... Not ideal at all for long term maintenance.

    Thanks

Reply
  • 0 in reply to Michael Dunn

    1) yes it is, we need to filter all outgoing traffic for security reasons.

    2) amazonaws.com was just an example, we have more domains that we notice are producing this weird behaviour, for example newrelic.com and rubygems.org

    Reproducing is very easy, just installing aws cli tools and invoking aws s3 ls will result in a web request to some random Amazon IP, rather than a hostname. This is very annoying. For now we wrote some regex to capture a lot of Amazon CIDR ranges... Not ideal at all for long term maintenance.

    Thanks

Children
Unfiltered HTML