10.0.3 "Sophos Network Extension" process using 150% CPU

We are seeing the same issue across our entire fleet. After the update to 10.0.3 that was applied this morning, we are seeing CPU usage from the Sophos Network Extension reaching greater than 100%, and memory usage skyrocketing in some cases consuming all available memory, and then the process crashing. 

We are also seeing significant issues with web socket based communications. As a result we've had to disable Endpoint Protection and InterceptX across our entire fleet. 

This appears to be the same experience I am having. I did not notice the memory leak until you pointed it out.

It turns out that in my case, the Network Extension eventually consumes all available memory and once it does, the process crashes. It eventually restarts and begins spiking the cpu and consumes all of the available memory until it crashes again. This is that loop I referred to in my original post where it would go for 20 minutes or more and then stop.

Hi mscottblake and Craine Runton,

Sorry to hear that you're having trouble. What issues are you seeing with web socket based communications? We would like to investigate the issue more, could you please provide a SDU from the affected machine? We would also like a sample of the affected process when the issue occurs.

To create a sample:

• Go into Activity Monitor, and double click on the affected process taking CPU usage (i.e.: Sophos Network Extension).
• Click on sample button
• Please private message me and attach the sample’s output text file

Please also provide an SDU from the affected machine by following these steps:

• Go into Central, find the device, and click on the generate SDU button
  • For more details, please refer to: https://support.sophos.com/support/s/article/KB-000038603?language=en_US
• Once the SDU is uploaded, you may post the file name here (or PM me) so we can extract it and investigate

Thanks for your feedback.

Hi Eric. I have a number of SDUs that I generated locally while troubleshooting and testing different configurations. I can provide the ZIP files via PM if you like. I will also generate some process samples for you of the Sophos Network Extension process while it is undergoing exponential memory growth. 

It's worth noting that at one point in my testing I had the Sophos Network Extension process using 17.94 GB of memory before it crashed. This is notable because my machine only has 16 GB of memory installed, and caused the system to use 8GB of swap to accommodate, which had crushing implications for my other running processes. 

As for the use of web sockets, my users have many issues using a variety of web services, such as Slack and Google Mail/Drive, whether through a native client or not. This is manifested by the applications repeatedly having to reopen WS connections. See the following two screenshots from the dev console while accessing Slack from Safari. Prior to enabling Malicious Traffic Detection, there was a single, long-lived socket connection. Afterwards, the socket had to continuously respawn, as shown below. 

Even this support forum isn't immune (though inspection seems to show this as being AJAX polling and not web sockets, but that points to a wider problem I suppose)

Additionally, our business is a software defined access platform whose local GUI connects to the local daemon over web sockets, and even that gets hammered by Sophos Network Extension even though it's all local machine traffic, We have had a number of customers who also use Sophos, and can confirm that they've had to disable Sophos to resume operations with our client. 

Sophos, have you made any progress on determining root cause for web sockets? In case it helps, I've noticed very similar problems with Carbon Black's web filter component. I stumbled upon this thread in the dev forums where Apple acknowledged there being issues with certain network filtering configurations https://developer.apple.com/forums/thread/667962. It's unclear if they've been solved. I am on 11.3 Beta and I still experience issues, so my guess is not quite.

Sophos, have you made any progress on determining root cause for web sockets? In case it helps, I've noticed very similar problems with Carbon Black's web filter component. I stumbled upon this thread in the dev forums where Apple acknowledged there being issues with certain network filtering configurations https://developer.apple.com/forums/thread/667962. It's unclear if they've been solved. I am on 11.3 Beta and I still experience issues, so my guess is not quite.