Thread Info
  • State Verified Answer
  • +2 person also asked this people also asked this
  • Locked Locked
  • Replies 27 replies
  • Answers 1 answer
  • Subscribers 6 subscribers
  • Views 73440 views
  • Users 0 members are here
Options
Suggested
This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

High bandwidth usage

Ross86
Hi all,

Yesterday we started getting high bandwidth usage showing on the external interface, almost all bandwidth taken.  It's coming from cloudfront.net servers and all I can see is the external interface is the client and its unclassified traffic on tcp/80.

I've throttled it based on the cloudfront IP ranges to stop the impact.

I can find a way to see if it's a user doing something as the logs show no high activity from specific internal clients.

Any idea what can be done to track this down?

Thanks


This thread was automatically locked due to age.
  • 0 in reply to William Warren
    You would disable it completely?  We only have a 20Mb leased line but are getting this increased to 50 or 100 soon.  This has all been in place without configuration changes for months so not sure why it's suddenly having an impact.
  • 0
    yes i would.  I bet your hit rate is less than 15% maybe even less than 10%.  Many sites are dynamic and are generate per view so there's very little to cache.  

    What i mean by a data cap is not the speed but the amount of data.  I am assuming you are in the us so you probably do not have a transfer cap.  with a hit rate so low you are simply wasting ram(which utm uses enough of anyway..the http proxy being the worst offender by far) and cpu cycles for checking the cache.  Plus by disabling the web cache you remove this high usage issue you hit.

    Owner:  Emmanuel Technology Consulting

    http://etc-md.com

    Former Sophos SG(Astaro) advocate/researcher/Silver Partner

    PfSense w/Suricata, ntopng, 

    Other addons to follow

  • 0
    Ok that makes sense.

    Oh ok no we don't have a data cap and I'm in the UK.  Thanks I'll disable cache if it continues.
  • 0
    Just an update:  I disabled cache and it hasn't helped, unclassified traffic from/to cloudfront.net is filling the interface.  When the cache is cleared it resolves it even though we don't have caching enabled, which is annoying as this when breaks web browsing until users reopen the browser and go to a http only site.

    Logged it with Sophos support who are analysing  some logs.
  • 0
    i got the same problem.

    Any news for that problem ?
  • 0

    We've had this same issue, twice in under a week. The external WAN interface maxes out our pipe ~100mbit, but we don't see anything close to the same amount of corresponding traffic/usage on any of the internal interfaces. It usually lasts about 30 minutes and makes any internet related activity basically unusable.

    The previous ideas about proxy/cache make sense, but has anybody figured out what specific application or process is causing it?

    SG310

    Firmware 9.408-4

  • 0 in reply to SteveFurness

    Steve, did you see my post on 8 Apr 2015 in this thread?

    Cheers - Bob

     
    Sophos UTM Community Moderator
    Sophos Certified Architect - UTM
    Sophos Certified Engineer - XG
    Gold Solution Partner since 2005
    MediaSoft, Inc. USA
  • 0 in reply to BAlfson

    I did, but there is no corresponding traffic on the internal interfaces, which leads me to believe that the client (A/V) isn't receiving the data. If the proxy doesn't have anywhere to send the traffic why/what does it keep downloading? The flow monitor doesn't give you a specific IP for where its connecting to/from, just the IP for our WAN interface and AWS.

    If this was just one connection that gets "stuck" downloading something, it shouldn't be noticeable to users. It would just use up whatever pipe is available, but when this happens everything else slows to a crawl, similar to if somebody was using BitTorrent and making thousands of connections that are all using as much available bandwidth as possible.

    Steve

     

  • 0 in reply to SteveFurness

    "I did, but there is no corresponding traffic on the internal interfaces, which leads me to believe that the client (A/V) isn't receiving the data." Exactly.  That's the phenomenon I was describing.  The problem is that the sending server times out so the Proxy restarts the download.

    Cheers - Bob

     
    Sophos UTM Community Moderator
    Sophos Certified Architect - UTM
    Sophos Certified Engineer - XG
    Gold Solution Partner since 2005
    MediaSoft, Inc. USA
  • +1 in reply to SteveFurness

    Do you have several hits for 'deferred download status refresh timeout, removing' when you search your Websecurity logs?

    I had one customer looping a download last week until the data partition was nearly full of broken downloads. After getting the hint to search for that phrase in websecurity logs I was able to identify WHAT was tried to be downloaded and was never successful I could create an exception to stop that bevaviour.

    I could only get rid of the nearly 15G 'trash downloads' by clearing the proxy cache (also caching is not even active on that UTM).

    Gruß / Regards,

    Kevin
    Sophos CE/CA (XG+UTM), Gold Partner

Unfiltered HTML