Google & "UDP flood" action

Hello

So to get straight to the point, I'm running Sophos UTM (FW Ver.: 9.203-3, Virtual) Home License and, as the thread title shows, browser-based Google products are affected by the IPS and some of its traffic are being tagged by the IPS as "UDP flood" firewall rule 60013, which is to Drop UDP_FLOOD attempts

As a result, some Google products will be capped at 2mbps download speeds. Strangely enough, it only happens to one wired client and not any virtualised clients nor wireless clients (so this means neither the LG Smart TV running YouTube nor the Android smartphone running Google Maps experience this issue.) When this plays out:

  • YouTube will load videos at 2mbps, causing buffer to 1080p videos and less often to 720p videos; and
  • Google Maps will load its chunks of map and image data slowly


Now, I can definitely turn off UDP flood protection, but that leaves a gigantic gap on my network. It's probably not the best practice when the UTM is responsible as the gateway between the Internet and my network at home. You now understand why this is probably something I would consider to avoid. I had disabled it for a minute and it definitely increased the loading speeds to what my ISP provides, which is 30x more than what it was throttling me to. As of right now, it's enabled.

Has anyone else experienced this issue? Has anyone found a fix for this? This started happening probably around the time where the OpenSSL Heartbleed vulnerability was discovered, if not a month or two before it.

UPDATE: Alright, so here's what's getting hit by IPS so that we all have that general idea...

IP of Google is the source IP. 
WAN IP is the destination.
Action is UDP Flood
Source Port: 443
Destination Port: Some random port on the 50000~60000s. 

Last time I checked, 443 isn't exactly UDP for the nature of what's being transported and a corporation like Google would keep atop for any such UDP floods to prevent it from happening.

Second Update
:
Please don't tell me this is too difficult. It has to be some simple explanation.
2014:07:14-16:40:20 core ulogd[4786]: id="2105" severity="info" sys="SecureNet" sub="ips" name="UDP flood detected" action="UDP flood" fwrule="60013" initf="eth1" srcmac="[Source MAC]" dstmac="[Destination MAC]" srcip="206.111.13.173" dstip="[My WAN IP]" proto="17" length="1228" tos="0x00" prec="0x00" ttl="57" srcport="443" dstport="55971"


Thanks
  • In reply to mork:

    Great way to post an issue - you anticipated the first question and answered it already!

    srcport="443" => Just clone that Service, change the name of the new one to, for example, "QUIC Responses," exchange the contents of Source and Destination and add the Service to your Exception.

    Note that this new service was the one I recommended earlier in the thread.  I've not seen a situation that requires an Exception for QUIC requests, but it probably doesn't hurt anything for you to leave it in place.

    Cheers - Bob

  • In reply to BAlfson:

    OMG, I was that focused on the networks and UDP stuff, that I didn't saw the obvious part of it.

    Thanks, a lot for opening my eyes, Bob :)


    Regards,
    Michael

  • Hello

    I have found the solution for this, just add these google addresses as an exception to udpflood

    • ip4:64.18.0.0/20
    • ip4:64.233.160.0/19
    • ip4:66.102.0.0/20
    • ip4:66.249.80.0/20
    • ip4:72.14.192.0/18
    • ip4:74.125.0.0/16
    • ip4:108.177.8.0/21
    • ip4:173.194.0.0/16
    • ip4:207.126.144.0/20
    • ip4:209.85.128.0/17
    • ip4:216.58.192.0/19
    • ip4:216.239.32.0/19
    • ip6:2001:4860:4000::/36
    • ip6:2404:6800:4000::/36
    • ip6:2607:f8b0:4000::/36
    • ip6:2800:3f0:4000::/36
    • ip6:2a00:1450:4000::/36
    • ip6:2c0f:fb50:4000::/36

     

     

     

    Regards

     

     

  • In reply to Moises Campos:

    Hi, Moises, and welcome to the UTM Community!

    Your solution will work perfectly, but I prefer to use the UDP 443 Exception as that makes the Exception more specific.

    Cheers - Bob

  • In reply to BAlfson:

    You are Right

     

    Thank you

  • In reply to BAlfson:

    Sorry to resurrect an old thread.  Seems there's two schools of thought to dealing with this issue.  Either exempt port 443 udp in either direction, or add a bunch of hosts to the exemption along with the service port.  Neither is an ideal solution because the first opens that port up to potential flooding from any external ip (?), while the latter has constantly changing hosts.

    I suppose the former (exempt the service port only) is the better of the two in that this exemption is valid for requests initiated behind the firewall.  Uninitiated external port 443 requests will still be blocked.  Bob am I correct in this understanding?

  • In reply to Jay Jay:

    Hi Jay,

    that is Google’s QUIC protocol. Close outgoing udp port 443. This protocol bypass the proxy, don't allow it in a productive environment. When closing the udp port chrome is going to use tcp.

    Google’s QUIC protocol:

    https://ma.ttias.be/googles-quic-protocol-moving-web-tcp-udp/

    Regards

    mod

  • In reply to mod2402:

    Agreed with the MoD, but I also prefer to add UDP 443 to the 'Allowed Target Services' in Web Filtering. That allows browsers using the explicit proxy to benefit from the increased throughput of https over UDP.   Using this means that you have the block as mod prescribes and the Anti-UDP Flooding Exception as described above.  In any case, there's no reason to open UDP 443 inbound.

    Cheers - Bob

  • In reply to BAlfson:

    Hi Bob,

    I've never though about that. Do you have experience with UDP 443 over the proxy? I've never tested that.

    regards

    mod

  • In reply to BAlfson:

    Bob, so to clarify, what does this rule look like in "Allowed Target Services" ?

    Also, by having it listed as an exception in intrusion prevention, doesn't the firewall still block inbound 443/udp requests?  Or because it's listed as an allowed service, it never makes it to the firewall which has no inbound 443/udp rule defined?

  • In reply to Jay Jay:

    Intrusion Prevention only happens to traffic allowed in.  It's unlikely that you have a web server using UDP 443, so no such traffic will even be seen by the Exception.

    As #2 in Rulz explains, the implicit Allow rule in Web Filtering is used before the manual firewall Block rule is considered.

    Cheers - Bob

  • In reply to BAlfson:

    Bob, with the rule set up as above I'm still getting massive IPS flood detection in the ips log on downloads with resulting speeds ~2-3 mbps.  For uploads there are no flood entries in the ips log and speeds are good, ~400-700 mbps.  Test is done using google drive in windows chrome browser to upload and dl large files.

    So it seems at least outbound the rule is properly respected but not inbound. 

    It almost seems like there needs to be connection tracking helper for this to work right to establish an inbound udp connection.

    Of course at this point, the best solution seems to be to just block udp/443 outbound altogether so chrome falls back on tcp.

    Also, I do want to point out, the issue isn't only with chrome browser.  For some time now i've had really slow google photo uploads (300-400 KBps).  This is related to this topic because once udp/443 out is blocked, uploads are happening at 7-10 MBps (megabytes/sec).

    Google drive on android seems to be stuck to around 2-3 MB/s in either direction regardless of how 443/udp is configured.

  • In reply to Jay Jay:

    "still getting massive IPS flood detection "

    UDP 443 requests have source ports in 1:65535.  UDP 443 responses have 443 as the source port and destination ports in 1:65535.  You can help future visitors here by showing a picture of the Edits of your Exception and of the UDP service you defined.

    If Web Filtering is in Transparent mode, I would block UDP 443 in a firewall rule.

    Cheers - Bob

  • In reply to BAlfson:

    Bob,

    Web filtering is in transparent mode.

    Here's the rules I have defined for allowed target service.

    This works fine for uploads but floods IPS with UDP flood notices when downloading.

    To eliminate use of QUIC entirely I have this firewall rule.

    Had to create separate service objects because allowed target service would not accept a UDP only definition. TCP/UDP was accepted.

    Are you saying QUIC would work correctly with the allowed target services entries if web filtering was off or not in transparent mode?

  • In reply to Jay Jay:

    The article mentioned earlier was certainly helpful.

    The most startling point was the observation that QUIC  provides a replacement for TLS 1.2.    Very smart people have performed tremendous research into TLS in recent years, producing a long strain of bad news that has forced us to migrate to TLS 1.2 from everything older, to discard MAC algorithms, and to discard older ciphers.    Who is vetting the QUIC encryption layer -- just Google?   Do business users really want to assume that Google has figured all of this out?

    Also based on the article, it appear that QUIC will always bypass https inspection (decrypt-and-scan) in the UTM web proxy, and probably most other brands as well.

    For file transfer applications like Google Drive, I wonder if the performance benefit is noticeable, other than on cell phones. File transfers imply long session, and TCP provides larger packet sizes.   At some point it would seem that the larger packet size would dominate the initial setup.   TCP might lose if the connection delivers a lot of packets out of order.

    The comments about UDP flood alarms suggest that by its very nature, QUIC makes connection tracking at the firewall into a very difficult task.   Since the plan is to update the protocol frequently, firewalls will have this problem on a continuing basis.

    I think I will block UDP 443 and live with the less optimal performance.