This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

WAF - a couple of learning points (learnt the hard way)

Last year, I replaced some clustered Cisco ASA 5510's with some clustered SG310's.

There was a fair amount of work to do as you simply could not export the rules and the terminology was a bit different. Nonetheless, the SG310's went in smoothly and we've been in the adhoc process of adding more and more functionality to users that the UTM's offer.

Part of this is WAF which we are using to retire the last ASA 5510 out as this is solely being used as a clientless SSL vpn (using java)

So I've been coming back to this a bit at a time as we discovered issues that took a little thinking about. The main root of our problems was DNS although you probably wouldn't have thought it was this looking at the errors.

The two big mistakes I made was:

1. Apply request routing for DNS for our external domain. As it was a new concept to us, we applied this from the UTM to all of our internal domain namespaces which was ok at the time for us to do due to the way our DNS was previously setup. We now have it setup so internal clients > internal dns servers > UTM. The UTM is the only device to perform recursive requests to the web.
The big mistake at the time was to apply DNS request routing  to our external domain from the UTM.

Our internal network is like any other eg 6 DNS servers with the internal domain and the external domain as well so that any external clients trying to get to serverA.mydomain.com would be able to use the internal ip address when they were inside the network and use the external public ip address when they were outside the network.

By getting the UTM to request route to the external DNS namespace on our internal DNS servers, the UTM was resolving serverA.mydomain.com to the internal ip address when it should have been resolving it to the external public ip address.

This had a massive detrimental effect on WAF. It was getting confused with the FQDN's. This wasn't noticed until we attempted to implement WAF

So, think very carefully when applying request routing on DNS and never "request route" the external domain name which you want WAF to use.

2. The 2nd issue was again a DNS issue and the mistake here was to add additional external hostnames to a host entry/definition on the UTM. So the UTM would see our internal  web server as an internal private ip but resolve the hostnames of the internal as well as the external hostname. Again, this caused confusion for the poor old WAF which by now, didn't know whether it was coming or going.

Bottom line is when using WAF (or the UTM) is to make sure it doesn't get confused with internal hostnames and external hostnames. So if you have an external FQDN pointing to an internal server, make sure that the UTM can only resolve the external FQDN to the external public ip and the internal server FQDN to the internal private ip. If it can resolve the external to the internal and vice versa, it will get confused and offer up mixed results that sometime work and sometimes don't.

Hopefully this will help somebody in the future and I think the moral of the story is:

The UTM offers a whole host of functionality but please be 100% sure as to what they mean before applying them otherwise you could end up with unpredictable behavior.



This thread was automatically locked due to age.
Parents
  • Pont 2 is very interesting.  I have an internal website that isn't loading CSS files or javascript when published for public access through WAF.  The Web Dev advised that those links were loading as the internal resources using the ip address, ie:@import url("WEBSERVERIP/.../comment.css instead of @import url("DOMAIN/.../comment.css certificate is for the domain, so it doesn’t load the css.  So far this only seems to happen on this Word Press site but I've only tested 3 sites so far.

    I'm wondering if this could be our issue?  If I do a DNS lookup on the UTM for the FQDN of the site it does resolve to the internal webserver IP.  UTM is primarily used as an AD integrated forward proxy so integrates with internal DNS.  This is an internal site that is accessible from internal clients so will resolve that FQDN from internal DNS.

    Do I need to create a host definition for this FQDN resolving to the public IP of the website?  Does mean that we'll have to revisit our browser proxy exceptions if we'll have to do this for every single published site because we have a lot of different website domains.  We are a Local Authority and it seems that services like to brand every single project that they run that needs a website e.g. go smarter, go green, go by bike....  I'm sure you get the idea  :(

  • Our DNS & UTM is set up like so:

    INTERNAL NETWORK
    4x DNS servers. All clients within the network point to these. On these dns servers, we have 2 zones, mydomain.local (our internal dns) and mydomain.com (our external dns)

    It is setup like this because we have mobile users and if we give them a shortcut to mydomain.local (internal resolution), when they are external, they can't resolve. So we give them links to mydomain.com which on our internal dns server points to a host with the internal ip eg 10.0.0.1
    When they leave the network and use their ISP's server, they will resolve to the external ip eg 1.2.3.4

    The internal dns servers forward to the UTM for external requests and only these are allowed access to the UTM dns proxy. That way nobody can resolve from within unless they go to our dns servers which in turn can only recursively resolve to the UTM.

    THE UTM setup

    Now the UTM might want to resolve to some internal clients. So we put in dns request routing for mydomain.local ONLY and not mydomain.com. Internal hosts should only ever have the internal name so the UTM doesn't resolve it to the external name.

    If you run a dns lookup on the UTM is should return the following:

    hostA.mydomain.local = 10.0.0.1 (because it is using request dns routing for mydomain.local)
    hostA.mydomain.com = 1.2.3.4 (because it can't resolve the hostname and will then recursively resolve to your external dns eg 1.2.3.4

    WAF

    When you set this up, the virtual webserver eg 1.2.3.4 points to your internal server 10.0.0.1
    It should not be allowed to mix this up with ie virtual webserver 1.2.3.4 points to 10.0.0.1 & 1.2.3.4

    If you think about it, you wouldn't DNAT your external ip 1.2.3.4 to an internal ip of 1.2.3.4 so I suppose it's the same principle. My WAF was getting mixed up with this and now it works fine. It's hard to spot because the additional hostnames are hidden but tell tale sign is to do a dns lookup from the UTM to see what it returns.

  • Unfortunately we can't segregate like that.  We have an internal AD local domain that's not publically visible but all of our non AD domains need to be accessed both publically and internally so we need to have zones in both internal and public DNS.  We'd never get away with setting up 2 domains for each website not least because we'd need to buy a SAN cert for every site but also because the officers in the business want their specific pretty domain name for their website.  By default our UTMs will resolve the internal IPs through internal DNS because it's configured as a backend firewall model and not an edge firewall in the same way as our TMGs are configured as a backend firewall.  It's also used primarily as a forward proxy/web filter solution.

    Yes we have virtual webservers set with a UTM listener IP and the real webservers have their own internal IP.  This is what worries me though.  This internal webserver IP should not be publically visible especially when using WAF and I don't know why this is the case.  I'm hoping that it's simply because the internal Word Press site is configured to use IPs in it's links but I don't know for certain.  I'm also not looking forward to tasking the web devs with trawling through all of their websites to look for links with webserver IPs in them  :(

  • Hi, when I said we had 2 domains, it was for simplicity's sake. You can have just 2 domains but 100 zones within dns that aren't in the slightest related to your AD eg internally, I can send anybody who types google.com to 10.1.0.1 if I want.

    The main thing here is to make sure that the UTM doesn't resolve to the internal domain name of the server you are publishing. Therefore, you can't have a host with the external hostname pointing to the internal ip. The internal host should only have the internal ip and internal hostname.

    If you need mobile users to resolve to an internal ip when inside the network, you simply create a zone in dns and become authoritative for it. That way, mobile users will always resolve to the internal ip of the host when within the network. As they move outside, your external dns server becomes authoritative for the domain and they resolve to the external address.

Reply
  • Hi, when I said we had 2 domains, it was for simplicity's sake. You can have just 2 domains but 100 zones within dns that aren't in the slightest related to your AD eg internally, I can send anybody who types google.com to 10.1.0.1 if I want.

    The main thing here is to make sure that the UTM doesn't resolve to the internal domain name of the server you are publishing. Therefore, you can't have a host with the external hostname pointing to the internal ip. The internal host should only have the internal ip and internal hostname.

    If you need mobile users to resolve to an internal ip when inside the network, you simply create a zone in dns and become authoritative for it. That way, mobile users will always resolve to the internal ip of the host when within the network. As they move outside, your external dns server becomes authoritative for the domain and they resolve to the external address.

Children
No Data