Advisory: Support Portal Maintenance. Login is currently unavailable, more info available here.

IPS service makes CPU up to 100% this morning

Dear All,

Since the support portal is still disable today, I only can share the issue over community.

This morning two of our customers found their XG's CPU was up to 100%.

And the network become unstable.





Stop the IPS service will resolve the issue.

But when restart IPS service, CPU is up to 100% again!

The firmware version is 17.5.14 MR-14.

My office's XG is this version too, but the issue didn't happen in my office.

Any suggestion?

Shunze



modify
[edited by: Shunze Lee at 3:18 AM (GMT -8) on 14 Jan 2021]
  • Hi Shunze,

    please check the IPS firmware versions to see if they are different to yours?

    Ian

     
    V18.0.x - e3-1225v5 6gb ram on 4 port MB with 2 x APX120 - 20w. 
    If a post solves your question use the 'This helped me' link.
  • Our XG's IPS version is 9.17.78.

    And the two customer's IP version are 9.17.78 too.

    But only the two customers have the issue.

  • Hi Shunze,

    let me see if one of the support guys will pickup their thread?

    h_patel would you please review Shunze's issue, thank you ?

    Ian

     
    V18.0.x - e3-1225v5 6gb ram on 4 port MB with 2 x APX120 - 20w. 
    If a post solves your question use the 'This helped me' link.
  • I have opened a case, ID: 03532672.
    But nobody reply me...

  • Checked my appliances, no issue at all. Does the pattern update date actually match with this date? 

    Can you reference the current Load on the gateway, is there any stream, going on, which IPS touches? Current activities - Live connection. Anything with a higher MB rate then usual. 

    __________________________________________________________________________________________________________________

  • I cannot report such issues from my systems but all are v18 MR4.

    What is your appliance hardware type?

    IPS and Application signatures
    18.17.78
    -
    18:26:36, Jan 13 2021

    other machine:

    Please be aware, that you will not receive pattern updates for IPS on below v18 MR1 appliances from tomorrow on!

  • Hello,

    This case has been escalated to a Senior Engineer for further investigation.

    Regards,


     
    Emmanuel (EmmoSophos)
    Community Support Engineer | Sophos Technical Support
    Sophos Support VideosProduct Documentation  |  @SophosSupport  | Sign up for SMS Alerts
    If a post solves your question use the 'Verify Answer' link.
  • Hi Shunze,

    we had this issue too on aXG210 18.01  MR-1-Build396

    Can you login to the advanced shell and tell us what processes consume the CPU?

    What we found out:

    1) Even if you set Detect and prevent:None on the rule(s) that might be responsible for the issue there is still some IPS activity
    2) You can try to disable IPS completely under System Services - Services - IPS (only recommendet for testing)
    3) You can bypass the IPS for indivdual rules on the CLI: https://support.sophos.com/support/s/article/KB-000038900?language=en_US

    The issue we had was that in a testing zone / testing networks (handled/configured somewhat like a DMZ) some devices were choosing the destination address randomly and sending UDP Traffic down to the default gateway which was the firewall. A continous stream of aprox. 300 MBit/s UDP Traffic was fired to an internal interface of the firewall which led to 100% CPU load on both processors. As the firewall also was the routing device the response times were very high for a LAN environment.

    Finally we removed the devices sending random addresses and decoupled the the testing network via a routing net. We did not fully test switching the IPS off completely (which is not a solution at all) and the rule exceptions as these things came up later.

    The traffic in the network of your customers make the difference not the firewall and most probably not the rules.

    So I'd also suggest you look in the network for malformed traffic (especially broadcasts) and devices that are doing weird things (wireshark, mirror port, drop rule).


    Best regards,
    Bernd