* Posting here as it touches multiple Sophos products *
We currently have a client that are experiencing significant issues since early last week whereby users of the VPN are being intermittently disconnected and a 'cypher' error detailed within the Sophos Connect client log.
For clarity, they have Sophos Intercept X deployed on all endpoints, and use Sophos Connect to access the corporate network via VPN. The VPN has a split tunnel configuration, with all traffic not destined for corporate internal IP addresses 'sent out' via their local internet breakout, with the traffic protected by iBoss Web Filtering.
Within the local network there have been no changes, and up until this point both Sophos and iBoss have been unable to identify root cause. From initial testing, disabling the iBoss Service appears to 'resolve' the issue. This has been fed back to iBoss, and in turn they said that 'there is an outstanding issue with iBoss and Sophos AV that we are waiting on Sophos to resolve'. Now I don't know how true this is, but is anyone else using this, or a similar, configuration and experiencing similar issues?
Advisory: Sophos Central Endpoint - Conflict between third-party web filter and Sophos web protection
We are experiencing similar issues, but we use the FortiClient VPN client. It seems to have started mid-week last week. Other things have also been affected, though, including:
Getting "Failed - Network error" when attempting to download large files from SharePoint Online/OneDrive (also accompanied by "Windows Defender SmartScreen is downloading from OneDrive" notification in Windows.
Getting errors when trying to open synced files in a user's OneDrive. One of the errors we've seen is "The cloud operation was not completed before the time-out period expired."
Getting a "Security Certificate was revoked" for outlook.office365.com message when opening Outlook
All these issues seem to be temporarily resolved by stopping the IBSA service, but only occur if Sophos is installed and updated. The versions we have observed are:
IBSA Version: 5.3.120 Sophos Core Agent: 2022.2.1.9 Sophos Intercept X: 2022.1.1.11 Sophos Endpoint Protection: 10.8.11.4 Sophos Device Encryption: 2022.1.0.58 Sophos XDR: 2022.2.1.9
Thank you Ian, this is very interesting. Have Sophos and/or iBoss been able to identify any root cause or provide guidance for further investigation?
I am awaiting contact from our Sophos Technical Account Manager to clarify the (unverified) claim from iBoss that there is a ‘known issue’ between the two software.
For sure! We just got this from our ticket with iBoss:
"Thank you for contacting iboss support. At this time our solution is to place the ibsa (iboss Windows connector service name) onto the Sophos global exclusion list. Additionally we recommend bypassing your iboss cloud gateway public IP addresses in Sophos as well. The devices will most likely need to be restarted for them to pull the latest update(s) from Sophos. Please let us know if you have any questions or concerns."
So far, it doesn't seem like iBoss's recommendations have fixed it for us.
The IP addresses you will need to add will be different for each iBoss deployment. You can find the IP addresses for the iBoss cloud nodes in your environment by reaching out to iBoss.
To add exclusions to Sophos for these IPs you'll need to add these under "Global Exclusions > Website (Windows/Mac)". You can also add exclusions in a Threat Protection Policy.
We have added all the cloud node IP addresses listed in our iBoss cloud account under Node Collection Management, but that doesn't seem to have made a difference.
Ian, I've had confirmation from our Technical Account Manager that there is a known issue between iBoss and Sophos. Sophos' development team are currently working on a resolution, but there are apparently two workarounds available. One has been shared higher in this chain, the other I am waiting on details for. As soon as I have these I'll share them with you.
Yes, they escalated our case to the development team. The only things I've found that have solved the issues so far, though, are uninstalling Sophos (disabling services doesn't seem to do anything), or stopping the IBSA service. I've temporarily deployed a script to all our machines disabling the IBSA service and scheduling a task to set it back to Automatic in two weeks while we wait for a resolution from Sophos.
Out of interest, are you still experiencing the issue? If so, does disabling Network Threat Protection within Sophos ‘resolve’ the issue? We’ve been doing some testing and this doesn’t temporarily resolve it, so wondering if there is a further Sophos issue that feeds into their wider iBoss compatibility issue.
Yes, we are still experiencing the issue, and no, disabling network threat protection doesn't appear to resolve the issue. We have had to temporarily disable the iBoss service on all our machines, which is obviously not ideal. In my testing, even if I disabled all Sophos services via the endpoint client on a test machine, the issue persisted. The only thing I could do on the Sophos end of things that would temporarily resolve the issue was uninstall it from the test machine.
Actually, just discovered something that makes me feel a little silly. If you turn off the service via the endpoint agent (after turning off tamper protection), it doesn't do anything; HOWEVER, if you then go and stop the Sophos Network Protection Service via services.mmc (or CLI), that DOES temporarily resolve the issue. Not sure why I never thought to test that way.
Add us to the list that it's affecting. I've been living with, but decided to tackle it today. I have added the exceptions, but haven't tested fully. Would be nice if it was just fixed in the product itself.
If the workarounds are not effective, I suggest opening a case with our support team. A test build is available currently, but the same changes will also be included in the Core Agent 2022.3 release.